Sep 10 21:17:14.374585 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 21:17:14.374600 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 21:17:14.374610 ip-10-0-128-173 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 21:17:14.374900 ip-10-0-128-173 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 21:17:24.624148 ip-10-0-128-173 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 21:17:24.624162 ip-10-0-128-173 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot bda1054d67e549f1a05496b838f7f263 --
Sep 10 21:19:23.406075 ip-10-0-128-173 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 21:19:23.736465 ip-10-0-128-173 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:19:23.736465 ip-10-0-128-173 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 21:19:23.736465 ip-10-0-128-173 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:19:23.736465 ip-10-0-128-173 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 21:19:23.736465 ip-10-0-128-173 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 21:19:23.738393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.738303    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 21:19:23.744195 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744170    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.744195 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744192    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.744195 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744198    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744203    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744209    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744213    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744218    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744222    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744226    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744230    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744232    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744236    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744239    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744242    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744244    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744247    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744249    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744252    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744255    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744257    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744259    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744262    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.744347 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744265    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744267    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744270    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744273    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744276    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744279    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744285    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744290    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744295    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744298    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744301    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744304    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744307    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744310    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744313    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744316    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744320    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744322    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744325    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.744847 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744327    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744330    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744332    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744335    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744337    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744341    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744344    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744347    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744349    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744352    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744355    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744358    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744360    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744363    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744365    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744367    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744370    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744373    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744375    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744378    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.745311 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744380    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744383    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744385    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744388    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744390    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744392    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744395    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744399    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744401    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744404    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744406    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744409    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744411    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744414    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744416    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744418    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744421    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744423    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744426    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744428    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.745834 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744431    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744433    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744437    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744439    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744443    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744445    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744448    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744450    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744453    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744455    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744458    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744460    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744464    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744467    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744470    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744473    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744476    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.744479    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745187    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745194    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.746326 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745198    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745201    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745204    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745206    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745209    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745212    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745215    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745218    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745220    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745223    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745226    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745230    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745234    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745237    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745240    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745244    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745247    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745249    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745252    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745254    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.746825 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745257    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745260    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745262    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745265    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745267    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745270    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745274    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745276    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745279    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745281    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745286    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745289    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745292    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745295    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745297    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745299    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745302    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745305    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745307    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745310    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.747316 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745312    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745315    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745317    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745320    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745323    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745327    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745330    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745333    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745390    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745394    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745397    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745399    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745402    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745405    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745407    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745409    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745412    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745414    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745417    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.747811 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745420    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745422    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745427    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745430    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745435    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745437    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745440    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745443    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745445    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745448    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745450    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745453    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745456    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745458    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745460    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745463    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745465    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745468    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745471    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.748272 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745473    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745476    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745478    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745483    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745486    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745488    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745491    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745493    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745496    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745498    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745500    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745503    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745505    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745508    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745510    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745513    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745515    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745518    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745520    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745952    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.748729 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745959    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745962    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745965    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745968    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745971    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745973    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745976    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745979    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745982    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745984    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745987    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745989    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745992    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745994    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.745996    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746002    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746004    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746007    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746009    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746012    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.749233 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746015    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746017    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746020    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746022    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746025    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746029    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746032    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746035    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746038    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746041    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746043    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746046    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746048    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746051    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746053    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746056    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746058    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746061    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746063    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.749722 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746066    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746069    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746071    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746073    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746076    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746078    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746082    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746085    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746091    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746094    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746097    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746100    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746103    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746105    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746108    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746110    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746113    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746115    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746118    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746121    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.750203 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746123    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746126    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746129    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746132    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746134    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746136    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746139    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746141    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746144    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746146    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746149    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746151    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746153    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746156    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746158    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746161    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746164    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746166    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746169    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746171    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.750723 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746175    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746177    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746180    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746182    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746185    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746187    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746191    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746194    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746197    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746200    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746203    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746205    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746208    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746210    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746214    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746217    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746219    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746222    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746224    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746307    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 21:19:23.751223 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746348    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746354    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746359    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746363    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746366    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746370    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746374    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746377    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746380    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746383    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746387    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746390    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746394    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746397    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746400    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746404    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746407    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746412    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746414    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746417    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746420    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746423    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746427    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 21:19:23.751726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746430    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746433    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746436    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746440    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746443    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746446    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746449    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746452    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746455    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746458    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746461    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746463    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746466    2579 flags.go:64] FLAG: --enable-server="true"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746469    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746473    2579 flags.go:64] FLAG: --event-burst="100"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746476    2579 flags.go:64] FLAG: --event-qps="50"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746479    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746481    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746484    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746487    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746490    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746494    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746497    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746500    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746503    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 21:19:23.752289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746506    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746509    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746512    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746515    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746517    2579 flags.go:64] FLAG: --feature-gates=""
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746521    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746524    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746527    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746530    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746532    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746536    2579 flags.go:64] FLAG: --help="false"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746539    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-128-173.ec2.internal"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746542    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746545    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746548    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746551    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746554    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746557    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746559    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746562    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746565    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746568    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746570    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746573    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 10 21:19:23.752931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746576    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746579    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746581    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746584    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746588    2579 flags.go:64] FLAG: --lock-file=""
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746591    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746594    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746596    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746602    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746605    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746608    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746611    2579 flags.go:64] FLAG: --logging-format="text"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746613    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746616    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746619    2579 flags.go:64] FLAG: --manifest-url=""
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746622    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746632    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746634    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746638    2579 flags.go:64] FLAG: --max-pods="110"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746641    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746644    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746647    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746650    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746653    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746656    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 21:19:23.753534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746659    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746666    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746668    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746671    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746674    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746677    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746681    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746684    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746687    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746690    2579 flags.go:64] FLAG: --port="10250"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746693    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746696    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-077c23d266639d04d"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746700    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746703    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746706    2579 flags.go:64] FLAG: --register-node="true"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746709    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746712    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746715    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746718    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746721    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746724    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746727    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746730    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746733    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746735    2579 flags.go:64] FLAG: --runonce="false"
Sep 10 21:19:23.754196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746738    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746741    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746744    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746747    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746750    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746752    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746755    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746758    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746761    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746764    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746767    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746769    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746772    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746792    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746795    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746801    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746803    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746806    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746810    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746814    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746817    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746819    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746822    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746825    2579 flags.go:64] FLAG: --v="2"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746829    2579 flags.go:64] FLAG: --version="false"
Sep 10 21:19:23.754818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746833    2579 flags.go:64] FLAG: --vmodule=""
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746837    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.746841    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746958    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746962    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746965    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746968    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746982    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746985    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746988    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746991    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746993    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746996    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.746999    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747003    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747006    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747009    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747012    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747015    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.755420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747018    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747020    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747023    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747026    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747028    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747031    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747033    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747054    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747057    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747060    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747064    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747067    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747069    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747072    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747075    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747078    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747080    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747083    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747085    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747088    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.755909 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747092    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747095    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747099    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747101    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747105    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747109    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747112    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747114    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747117    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747120    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747123    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747125    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747128    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747131    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747133    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747136    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747139    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747141    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747144    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.756498 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747148    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747150    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747153    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747156    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747158    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747161    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747163    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747166    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747172    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747175    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747177    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747179    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747182    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747185    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747188    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747190    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747193    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747195    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747198    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747200    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.756995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747203    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747205    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747208    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747210    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747213    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747215    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747218    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747220    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747223    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747225    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747228    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747230    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747234    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747236    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747239    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747242    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747244    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747247    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747249    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747252    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.757524 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747254    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747257    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747260    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.747262    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.747802    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.755090    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.755112    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.755991    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.755998    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756053    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756058    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756061    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756065    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756068    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756071    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.758042 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756074    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756076    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756079    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756082    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756085    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756087    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756090    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756092    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756095    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756098    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756100    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756103    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756105    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756108    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756111    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756114    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756116    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756119    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756125    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756128    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.758411 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756131    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756134    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756136    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756139    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756142    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756145    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756148    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756151    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756153    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756156    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756158    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756161    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756164    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756166    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756169    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756171    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756174    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756176    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756179    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756181    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.759005 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756184    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756187    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756189    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756192    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756194    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756197    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756200    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756202    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756204    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756207    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756209    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756212    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756215    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756217    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756220    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756223    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756226    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756229    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756231    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.759499 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756234    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756236    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756239    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756242    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756244    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756246    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756249    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756251    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756254    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756256    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756259    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756261    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756264    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756266    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756269    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756271    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756274    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756276    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756279    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756281    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.759969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756284    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756286    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756289    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756292    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756296    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756300    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756303    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756308    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756313    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756316    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756319    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756322    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756325    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756328    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.760446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.756333    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756448    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756452    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756455    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756458    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756461    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756464    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756467    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756470    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756473    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756476    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756478    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756481    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756483    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756486    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756488    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756491    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756493    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756496    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756498    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 21:19:23.760812 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756501    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756504    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756506    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756509    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756512    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756515    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756518    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756521    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756524    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756527    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756531    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756534    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756537    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756539    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756542    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756544    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756547    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756549    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756552    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 21:19:23.761304 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756554    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756556    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756559    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756561    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756564    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756567    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756569    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756571    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756574    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756576    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756578    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756581    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756584    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756586    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756588    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756591    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756593    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756595    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756598    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756601    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 21:19:23.761763 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756604    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756607    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756609    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756612    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756614    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756617    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756619    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756621    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756624    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756626    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756629    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756631    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756634    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756636    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756639    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756643    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756646    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756649    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756652    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756655    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 21:19:23.762267 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756658    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756660    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756662    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756665    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756668    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756670    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756673    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756676    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756680    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756682    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756685    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756688    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756691    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756693    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756696    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756699    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756701    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756703    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756706    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756709    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 21:19:23.762747 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:23.756711    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.756716    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.756834    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.758841    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.759592    2579 server.go:1019] "Starting client certificate rotation"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.759693    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.759727    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.761897    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:19:23.763310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.763049    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 21:19:23.769808 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.769790    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 10 21:19:23.772387 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.772372    2579 log.go:25] "Validated CRI v1 image API"
Sep 10 21:19:23.773255 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.773241    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 21:19:23.777272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.777251    2579 fs.go:135] Filesystem UUIDs: map[032ac829-84f4-473d-a039-fe3db71b6b67:/dev/nvme0n1p4 72d0e87d-850c-4f23-9138-bcf1f60f7895:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 10 21:19:23.777356 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.777270    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 21:19:23.782120 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.782006    2579 manager.go:216] Machine: {Timestamp:2026-09-10 21:19:23.780844441 +0000 UTC m=+0.275667970 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3139169 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ad4f1e1fff5dfb60cb0e075359387 SystemUUID:ec2ad4f1-e1ff-f5df-b60c-b0e075359387 BootID:bda1054d-67e5-49f1-a054-96b838f7f263 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e7:da:73:11:f1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e7:da:73:11:f1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:57:76:15:cc:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 21:19:23.782120 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.782109    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 21:19:23.782280 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.782211    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 21:19:23.785048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.785022    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 21:19:23.785209 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.785050    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 21:19:23.785674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.785664    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 21:19:23.785708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.785676    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 21:19:23.785708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.785688    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:19:23.786214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.786204    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 21:19:23.786999 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.786990    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 21:19:23.787455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.787438    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:19:23.790932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.790921    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:19:23.791040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791032    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 21:19:23.791074 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791060    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 21:19:23.791101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791078    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 21:19:23.791101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791092    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 21:19:23.791101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791101    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 10 21:19:23.791187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791109    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 21:19:23.791473 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791461    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:19:23.791511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.791487    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 21:19:23.794384 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.794365    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 21:19:23.796654 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.796640    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 21:19:23.796699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.796678    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 21:19:23.798399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798385    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798404    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798410    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798418    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798432    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798440    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798455    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 21:19:23.798459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798460    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 21:19:23.798641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798467    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 21:19:23.798641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798473    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 21:19:23.798641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798481    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 21:19:23.798641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.798486    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 21:19:23.800233 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.800219    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 21:19:23.800233 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.800234    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 21:19:23.800710 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.800687    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 21:19:23.800710 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.800703    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 21:19:23.802835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.802817    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 21:19:23.803931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.803917    2579 server.go:1268] "Started kubelet"
Sep 10 21:19:23.804010 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.803989    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 21:19:23.804165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.804084    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 21:19:23.804247 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.804186    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 21:19:23.804732 ip-10-0-128-173 systemd[1]: Started Kubernetes Kubelet.
Sep 10 21:19:23.805848 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.805654    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 21:19:23.808101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.808081    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 21:19:23.810869 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.810850    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 21:19:23.811237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.811223    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9zdz2"
Sep 10 21:19:23.813064 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.813046    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 21:19:23.813694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.813679    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 21:19:23.813833 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.813815    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:23.814264 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.814248    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 21:19:23.814264 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.814265    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 21:19:23.814396 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.814248    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 21:19:23.814396 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.814366    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 21:19:23.814396 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.814371    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 21:19:23.815147 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815130    2579 factory.go:55] Registering systemd factory
Sep 10 21:19:23.815195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815152    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 10 21:19:23.815389 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815366    2579 factory.go:153] Registering CRI-O factory
Sep 10 21:19:23.815389 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815386    2579 factory.go:223] Registration of the crio container factory successfully
Sep 10 21:19:23.815509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815437    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 21:19:23.815509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815458    2579 factory.go:103] Registering Raw factory
Sep 10 21:19:23.815509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.815474    2579 manager.go:1196] Started watching for new ooms in manager
Sep 10 21:19:23.816349 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.816334    2579 manager.go:349] Starting recovery of all containers
Sep 10 21:19:23.816771 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.816728    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 21:19:23.817113 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.816942    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 21:19:23.817932 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.816681    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18d41261ef0128ba  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-09-10 21:19:23.803887802 +0000 UTC m=+0.298711331,LastTimestamp:2026-09-10 21:19:23.803887802 +0000 UTC m=+0.298711331,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Sep 10 21:19:23.819643 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.819618    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9zdz2"
Sep 10 21:19:23.824033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.824018    2579 manager.go:354] Recovery completed
Sep 10 21:19:23.829218 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.829206    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:23.831001 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.830985    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:23.831075 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831014    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:23.831075 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831025    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:23.831523 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831508    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 21:19:23.831523 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831522    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 21:19:23.831640 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831540    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 21:19:23.831640 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.831563    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:23.832224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.832210    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:23.832284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.832235    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:23.832284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.832247    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:23.833623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.833606    2579 policy_none.go:49] "None policy: Start"
Sep 10 21:19:23.833688 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.833633    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 21:19:23.833688 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.833646    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 21:19:23.833874 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.833546    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18d41261f09ee0e7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-09-10 21:19:23.831001319 +0000 UTC m=+0.325824853,LastTimestamp:2026-09-10 21:19:23.831001319 +0000 UTC m=+0.325824853,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Sep 10 21:19:23.835802 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.835719    2579 policy_none.go:47] "Start"
Sep 10 21:19:23.879354 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879335    2579 manager.go:337] "Starting Device Plugin manager"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.879385    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879399    2579 server.go:87] "Starting device plugin registration server"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879724    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879734    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879864    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879959    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.879969    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.880330    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 21:19:23.890541 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.880382    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:23.947877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.947847    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 21:19:23.949265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.949246    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 21:19:23.949333 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.949302    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 21:19:23.949333 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.949326    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 21:19:23.949427 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.949368    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 21:19:23.951687 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.951671    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:19:23.980677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.980655    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:23.981630 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.981615    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:23.981693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.981642    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:23.981693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.981652    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:23.981693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.981673    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-173.ec2.internal"
Sep 10 21:19:23.989815 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:23.989733    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-173.ec2.internal"
Sep 10 21:19:23.989815 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:23.989756    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-173.ec2.internal\": node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.006843 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.006824    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.050135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.050113    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"]
Sep 10 21:19:24.050212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.050169    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:24.050956 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.050934    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:24.051040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.050965    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:24.051040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.050976    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:24.053324 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.053312    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:24.053438 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.053424    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.053486 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.053456    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:24.054038 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054020    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:24.054128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054038    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:24.054128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054050    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:24.054128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054061    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:24.054128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054063    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:24.054128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.054078    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:24.056365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.056348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.056424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.056383    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 21:19:24.057063 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.057047    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 21:19:24.057155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.057068    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 21:19:24.057155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.057079    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Sep 10 21:19:24.070307 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.070290    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.074566 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.074552    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.107385 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.107369    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.116737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.116720    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2d6ebd71b368b317d437cef68944460-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"b2d6ebd71b368b317d437cef68944460\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.116810 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.116743    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.116810 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.116760    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.208017 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.207991    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.217424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2d6ebd71b368b317d437cef68944460-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"b2d6ebd71b368b317d437cef68944460\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.217491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.217491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.217586 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.217586 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217523    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f1e34c96776044f47f6bc80f4249aad8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"f1e34c96776044f47f6bc80f4249aad8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.217586 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.217528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b2d6ebd71b368b317d437cef68944460-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"b2d6ebd71b368b317d437cef68944460\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.308670 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.308648    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.373179 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.373142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.378369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.378024    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:24.409704 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.409680    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.510177 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.510144    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.610769 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.610712    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.711202 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.711175    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.759718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.759688    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 21:19:24.760132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.759847    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 21:19:24.811902 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.811887    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.824567 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.824537    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 21:14:24 +0000 UTC" deadline="2028-04-01 23:18:03.073308321 +0000 UTC"
Sep 10 21:19:24.824567 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.824565    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13657h58m38.248746622s"
Sep 10 21:19:24.835724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.835702    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:19:24.912647 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:24.912579    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:24.969370 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:24.969337    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d6ebd71b368b317d437cef68944460.slice/crio-b10d0bd947f2f7c7afca1d0c5e902094e2a1ec402b4731632fae8ab50f5834d6 WatchSource:0}: Error finding container b10d0bd947f2f7c7afca1d0c5e902094e2a1ec402b4731632fae8ab50f5834d6: Status 404 returned error can't find the container with id b10d0bd947f2f7c7afca1d0c5e902094e2a1ec402b4731632fae8ab50f5834d6
Sep 10 21:19:24.969600 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:24.969578    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e34c96776044f47f6bc80f4249aad8.slice/crio-18527b5b58fb620a14cab1333486056b83d2cd1ed340c8b30236fd6415fea2a2 WatchSource:0}: Error finding container 18527b5b58fb620a14cab1333486056b83d2cd1ed340c8b30236fd6415fea2a2: Status 404 returned error can't find the container with id 18527b5b58fb620a14cab1333486056b83d2cd1ed340c8b30236fd6415fea2a2
Sep 10 21:19:24.973222 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:24.973206    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:19:25.012947 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.012926    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Sep 10 21:19:25.026999 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.026977    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:19:25.114014 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.113984    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:25.123432 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.123412    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:19:25.124203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.124191    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Sep 10 21:19:25.133268 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.133255    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 21:19:25.208360 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.208297    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:19:25.792242 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.792206    2579 apiserver.go:52] "Watching apiserver"
Sep 10 21:19:25.797866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.797835    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 21:19:25.798303 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.798271    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-l69zp","openshift-multus/multus-6w4fx","openshift-multus/multus-additional-cni-plugins-nkwbq","openshift-network-diagnostics/network-check-target-kntrm","openshift-ovn-kubernetes/ovnkube-node-kv2bd","kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn","openshift-image-registry/node-ca-ns59t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal","openshift-multus/network-metrics-daemon-k2tdn","openshift-network-operator/iptables-alerter-m5jfg","kube-system/global-pull-secret-syncer-xqrzz","kube-system/konnectivity-agent-bcwzb","openshift-cluster-node-tuning-operator/tuned-h8khf"]
Sep 10 21:19:25.801378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.801360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.804367 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.804332    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 21:19:25.804766 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.804743    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.809841 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.809817    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.810007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.809873    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.810007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.809761    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cjcs4\""
Sep 10 21:19:25.811297 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.811275    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.811855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.811834    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:25.811964 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.811937    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:25.812629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.812545    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gtj6t\""
Sep 10 21:19:25.812714 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.812687    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4wggk\""
Sep 10 21:19:25.812838 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.812820    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 21:19:25.813041 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813013    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 21:19:25.813108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 21:19:25.813253 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813237    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 21:19:25.813355 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813339    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 21:19:25.813412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813396    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.813558 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.813538    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.816323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.814263    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.816323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.814887    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 21:19:25.816323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.815912    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 21:19:25.816323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.815995    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.816323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.816157    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 21:19:25.816821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.816801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 21:19:25.817060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.817040    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.817150 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.817068    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bgsjv\""
Sep 10 21:19:25.817150 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.817127    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 21:19:25.817254 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.817131    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.818948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.818864    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7fcbw\""
Sep 10 21:19:25.819238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.819211    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.819490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.819472    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.819639 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.819620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:25.821582 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.821508    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.821869 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.821851    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:25.822223 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.822206    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.822334 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.822295    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8p5kl\""
Sep 10 21:19:25.822524 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.822500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 21:19:25.824081 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.824064    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4kn88\""
Sep 10 21:19:25.824287 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.824266    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.824610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.824590    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 21:19:25.824853 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.824818    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.825071 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.825055    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.827084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827064    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kqmkk\""
Sep 10 21:19:25.827309 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827298    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 21:19:25.827727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827707    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:25.827838 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.827759    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:25.827838 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827780    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.827838 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827813    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-systemd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827929    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-system-cni-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-netns\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.827989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-etc-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-bin\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-host\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-sys-fs\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-var-lib-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828197    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-netd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovn-node-metrics-cert\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-run\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-sys\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-lib-modules\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828322    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-device-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828344    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-kubelet\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-slash\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.828420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828398    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-env-overrides\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828461    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9dgj\" (UniqueName: \"kubernetes.io/projected/9c8523ff-06ac-4d0f-aa80-7646e3a74077-kube-api-access-v9dgj\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-systemd\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c5x8\" (UniqueName: \"kubernetes.io/projected/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kube-api-access-4c5x8\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cnibin\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5pj\" (UniqueName: \"kubernetes.io/projected/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-kube-api-access-wj5pj\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-systemd-units\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-modprobe-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828729    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-kubernetes\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d0cf93-576a-44e5-9fdc-3d7813d8e372-agent-certs\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-node-log\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-config\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-conf\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.829325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-var-lib-kubelet\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.828992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-tuned\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-registration-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829056    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-ovn\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-script-lib\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysconfig\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829126    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-tmp\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829150    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r468\" (UniqueName: \"kubernetes.io/projected/d00af7d9-fc18-43ef-b3ba-01279ef301bb-kube-api-access-9r468\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829201    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-log-socket\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829257    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-socket-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.829283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d0cf93-576a-44e5-9fdc-3d7813d8e372-konnectivity-ca\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.830092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.830030    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:25.832047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.832027    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 21:19:25.832156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.832069    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jx2q9\""
Sep 10 21:19:25.832156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.832077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 21:19:25.832364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.832344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:25.832445 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.832412    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:25.832545 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.832507    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:19:25.847625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.847603    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5x5cj"
Sep 10 21:19:25.854971 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.854947    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5x5cj"
Sep 10 21:19:25.915463 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.915441    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 21:19:25.929571 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pccc\" (UniqueName: \"kubernetes.io/projected/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-kube-api-access-8pccc\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.929699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-dbus\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:25.929699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:25.929699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a5bf0dc-7f92-4686-bd79-928113c62b59-host-slash\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:25.929699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5pj\" (UniqueName: \"kubernetes.io/projected/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-kube-api-access-wj5pj\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929722    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-modprobe-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-tuned\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cni-binary-copy\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-k8s-cni-cncf-io\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-multus\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdtwq\" (UniqueName: \"kubernetes.io/projected/5a5bf0dc-7f92-4686-bd79-928113c62b59-kube-api-access-mdtwq\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:25.929946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929962    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-socket-dir-parent\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.929987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/763c18c2-136b-41a3-b05d-4907bf940f7b-hosts-file\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-modprobe-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-ovn\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-script-lib\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-ovn\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-bin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtmsg\" (UniqueName: \"kubernetes.io/projected/3f478cf5-0402-452a-8a3d-981ed11e30cf-kube-api-access-dtmsg\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-kubelet-config\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:25.930318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-daemon-config\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930377    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-socket-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-socket-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d0cf93-576a-44e5-9fdc-3d7813d8e372-konnectivity-ca\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-systemd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930576    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930607    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6388b71-4b19-4847-a45a-8c86d80b9bcd-host\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930622    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763c18c2-136b-41a3-b05d-4907bf940f7b-tmp-dir\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930661    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-netns\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930744    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-etc-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-bin\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.930796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-kubelet\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scdcb\" (UniqueName: \"kubernetes.io/projected/763c18c2-136b-41a3-b05d-4907bf940f7b-kube-api-access-scdcb\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930892    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovn-node-metrics-cert\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930913    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-run\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930977    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-sys\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-script-lib\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931107    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-run\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-netns\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931159    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.930983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-sys\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv8q5\" (UniqueName: \"kubernetes.io/projected/a6388b71-4b19-4847-a45a-8c86d80b9bcd-kube-api-access-sv8q5\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931259    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-device-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-slash\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-env-overrides\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.931443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-hostroot\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c5x8\" (UniqueName: \"kubernetes.io/projected/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kube-api-access-4c5x8\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cnibin\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-systemd-units\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-kubernetes\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931603    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-conf\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931635    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r468\" (UniqueName: \"kubernetes.io/projected/d00af7d9-fc18-43ef-b3ba-01279ef301bb-kube-api-access-9r468\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58d0cf93-576a-44e5-9fdc-3d7813d8e372-konnectivity-ca\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d0cf93-576a-44e5-9fdc-3d7813d8e372-agent-certs\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-systemd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-node-log\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931738    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-node-log\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-config\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931773    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-systemd-units\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-var-lib-kubelet\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-multus-certs\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.934370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-device-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931876    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-slash\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931880    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-registration-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysconfig\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-run-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-tmp\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-d\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6388b71-4b19-4847-a45a-8c86d80b9bcd-serviceca\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.931994    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-run-ovn-kubernetes\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-etc-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-bin\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-log-socket\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cnibin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5a5bf0dc-7f92-4686-bd79-928113c62b59-iptables-alerter-script\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-system-cni-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-host\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932352    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-system-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-sys-fs\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-var-lib-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-netd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-lib-modules\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-netns\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-conf-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-env-overrides\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932611    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-etc-kubernetes\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-cni-netd\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-host\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.935923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-etc-selinux\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932717    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-lib-modules\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysctl-conf\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932755    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-kubelet\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9dgj\" (UniqueName: \"kubernetes.io/projected/9c8523ff-06ac-4d0f-aa80-7646e3a74077-kube-api-access-v9dgj\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-systemd\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932931    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-systemd\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.932978    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-host-kubelet\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-kubernetes\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933043    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-sysconfig\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d00af7d9-fc18-43ef-b3ba-01279ef301bb-var-lib-kubelet\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cnibin\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-registration-dir\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-var-lib-openvswitch\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933468    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-system-cni-dir\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.933941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c8523ff-06ac-4d0f-aa80-7646e3a74077-log-socket\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.936625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.934015    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-cni-binary-copy\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.937285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.934033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-sys-fs\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.937285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.934305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovnkube-config\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.937285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.935992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-etc-tuned\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.937285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.936028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d00af7d9-fc18-43ef-b3ba-01279ef301bb-tmp\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.937285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.936295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58d0cf93-576a-44e5-9fdc-3d7813d8e372-agent-certs\") pod \"konnectivity-agent-bcwzb\" (UID: \"58d0cf93-576a-44e5-9fdc-3d7813d8e372\") " pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:25.937735 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.937715    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c8523ff-06ac-4d0f-aa80-7646e3a74077-ovn-node-metrics-cert\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.942081 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.942058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5pj\" (UniqueName: \"kubernetes.io/projected/fb3a5920-8efd-4b31-9aa5-03ff83238bd9-kube-api-access-wj5pj\") pod \"multus-additional-cni-plugins-nkwbq\" (UID: \"fb3a5920-8efd-4b31-9aa5-03ff83238bd9\") " pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:25.943487 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.943465    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:25.943610 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.943496    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:25.943610 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.943512    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:25.943610 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:25.943585    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:26.443559047 +0000 UTC m=+2.938382587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:25.945038 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.945002    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9dgj\" (UniqueName: \"kubernetes.io/projected/9c8523ff-06ac-4d0f-aa80-7646e3a74077-kube-api-access-v9dgj\") pod \"ovnkube-node-kv2bd\" (UID: \"9c8523ff-06ac-4d0f-aa80-7646e3a74077\") " pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:25.946318 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.946299    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c5x8\" (UniqueName: \"kubernetes.io/projected/eae78f6f-4ca0-4651-bdef-b60eb4b84d21-kube-api-access-4c5x8\") pod \"aws-ebs-csi-driver-node-qjfkn\" (UID: \"eae78f6f-4ca0-4651-bdef-b60eb4b84d21\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:25.946405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.946348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r468\" (UniqueName: \"kubernetes.io/projected/d00af7d9-fc18-43ef-b3ba-01279ef301bb-kube-api-access-9r468\") pod \"tuned-h8khf\" (UID: \"d00af7d9-fc18-43ef-b3ba-01279ef301bb\") " pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:25.955738 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.955691    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"b2d6ebd71b368b317d437cef68944460","Type":"ContainerStarted","Data":"b10d0bd947f2f7c7afca1d0c5e902094e2a1ec402b4731632fae8ab50f5834d6"}
Sep 10 21:19:25.957112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.957087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"f1e34c96776044f47f6bc80f4249aad8","Type":"ContainerStarted","Data":"18527b5b58fb620a14cab1333486056b83d2cd1ed340c8b30236fd6415fea2a2"}
Sep 10 21:19:25.999722 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:25.999697    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 21:19:26.033519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-hostroot\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-multus-certs\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6388b71-4b19-4847-a45a-8c86d80b9bcd-serviceca\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cnibin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5a5bf0dc-7f92-4686-bd79-928113c62b59-iptables-alerter-script\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-hostroot\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-multus-certs\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.033645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cnibin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033630    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-system-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033689    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-system-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-netns\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-netns\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033735    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-conf-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-etc-kubernetes\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033787    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-cni-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pccc\" (UniqueName: \"kubernetes.io/projected/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-kube-api-access-8pccc\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033843    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-dbus\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033849    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-conf-dir\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a5bf0dc-7f92-4686-bd79-928113c62b59-host-slash\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cni-binary-copy\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033941    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-k8s-cni-cncf-io\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-multus\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.033957    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034014    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-dbus\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.034084 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.034019    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:26.53400299 +0000 UTC m=+3.028826507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdtwq\" (UniqueName: \"kubernetes.io/projected/5a5bf0dc-7f92-4686-bd79-928113c62b59-kube-api-access-mdtwq\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.033862    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-etc-kubernetes\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-socket-dir-parent\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a6388b71-4b19-4847-a45a-8c86d80b9bcd-serviceca\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034135    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-multus\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034147    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/763c18c2-136b-41a3-b05d-4907bf940f7b-hosts-file\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034168    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5a5bf0dc-7f92-4686-bd79-928113c62b59-host-slash\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034179    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-run-k8s-cni-cncf-io\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-bin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-cni-bin\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/763c18c2-136b-41a3-b05d-4907bf940f7b-hosts-file\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-socket-dir-parent\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.034304    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtmsg\" (UniqueName: \"kubernetes.io/projected/3f478cf5-0402-452a-8a3d-981ed11e30cf-kube-api-access-dtmsg\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-kubelet-config\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.034721 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.034351    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:26.53433507 +0000 UTC m=+3.029158604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-daemon-config\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5a5bf0dc-7f92-4686-bd79-928113c62b59-iptables-alerter-script\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6388b71-4b19-4847-a45a-8c86d80b9bcd-host\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763c18c2-136b-41a3-b05d-4907bf940f7b-tmp-dir\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-kubelet\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034458    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scdcb\" (UniqueName: \"kubernetes.io/projected/763c18c2-136b-41a3-b05d-4907bf940f7b-kube-api-access-scdcb\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034471    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6388b71-4b19-4847-a45a-8c86d80b9bcd-host\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034483    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv8q5\" (UniqueName: \"kubernetes.io/projected/a6388b71-4b19-4847-a45a-8c86d80b9bcd-kube-api-access-sv8q5\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034381    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-kubelet-config\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-cni-binary-copy\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-host-var-lib-kubelet\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034709    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763c18c2-136b-41a3-b05d-4907bf940f7b-tmp-dir\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.035388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.034941    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-multus-daemon-config\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.044422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.044402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtmsg\" (UniqueName: \"kubernetes.io/projected/3f478cf5-0402-452a-8a3d-981ed11e30cf-kube-api-access-dtmsg\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:26.049026 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.048998    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdtwq\" (UniqueName: \"kubernetes.io/projected/5a5bf0dc-7f92-4686-bd79-928113c62b59-kube-api-access-mdtwq\") pod \"iptables-alerter-m5jfg\" (UID: \"5a5bf0dc-7f92-4686-bd79-928113c62b59\") " pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.049128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.049105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv8q5\" (UniqueName: \"kubernetes.io/projected/a6388b71-4b19-4847-a45a-8c86d80b9bcd-kube-api-access-sv8q5\") pod \"node-ca-ns59t\" (UID: \"a6388b71-4b19-4847-a45a-8c86d80b9bcd\") " pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.050891 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.050870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pccc\" (UniqueName: \"kubernetes.io/projected/4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24-kube-api-access-8pccc\") pod \"multus-6w4fx\" (UID: \"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24\") " pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.056917 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.056894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scdcb\" (UniqueName: \"kubernetes.io/projected/763c18c2-136b-41a3-b05d-4907bf940f7b-kube-api-access-scdcb\") pod \"node-resolver-l69zp\" (UID: \"763c18c2-136b-41a3-b05d-4907bf940f7b\") " pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.123598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.123532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn"
Sep 10 21:19:26.131134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.131107    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:26.139105 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.139089    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nkwbq"
Sep 10 21:19:26.144848 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.144823    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:26.153833 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.153815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h8khf"
Sep 10 21:19:26.169728 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.169710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ns59t"
Sep 10 21:19:26.178122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.178097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l69zp"
Sep 10 21:19:26.186507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.186488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6w4fx"
Sep 10 21:19:26.194579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.194552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m5jfg"
Sep 10 21:19:26.537878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.537849    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:26.538065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.537924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:26.538065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.537960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:26.538065 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538042    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:26.538065 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538063    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538071    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538085    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538113    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538124    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:27.538101473 +0000 UTC m=+4.032925011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538144    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:27.538133667 +0000 UTC m=+4.032957192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:26.538259 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:26.538161    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:27.538152132 +0000 UTC m=+4.032975658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:26.858222 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.858179    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:14:26 +0000 UTC" deadline="2028-02-04 12:01:08.366629945 +0000 UTC"
Sep 10 21:19:26.858532 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.858222    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12278h41m41.508411777s"
Sep 10 21:19:26.875434 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.875404    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae78f6f_4ca0_4651_bdef_b60eb4b84d21.slice/crio-77dee46917a7255d09a9b7e17df97c053d62819d04bcdc3fadeee9dd6ed919a0 WatchSource:0}: Error finding container 77dee46917a7255d09a9b7e17df97c053d62819d04bcdc3fadeee9dd6ed919a0: Status 404 returned error can't find the container with id 77dee46917a7255d09a9b7e17df97c053d62819d04bcdc3fadeee9dd6ed919a0
Sep 10 21:19:26.876426 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.876396    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8523ff_06ac_4d0f_aa80_7646e3a74077.slice/crio-8564d82cfd765a9edb1199bf2b6cabd4b073186c7cf6c6e9ad046021e3bc70a5 WatchSource:0}: Error finding container 8564d82cfd765a9edb1199bf2b6cabd4b073186c7cf6c6e9ad046021e3bc70a5: Status 404 returned error can't find the container with id 8564d82cfd765a9edb1199bf2b6cabd4b073186c7cf6c6e9ad046021e3bc70a5
Sep 10 21:19:26.877816 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.877704    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00af7d9_fc18_43ef_b3ba_01279ef301bb.slice/crio-0733ad3f525318dc121a1b514fc36a69b359699b0d69b661e786ebeba1e115f4 WatchSource:0}: Error finding container 0733ad3f525318dc121a1b514fc36a69b359699b0d69b661e786ebeba1e115f4: Status 404 returned error can't find the container with id 0733ad3f525318dc121a1b514fc36a69b359699b0d69b661e786ebeba1e115f4
Sep 10 21:19:26.881464 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.881443    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8e000e_c4fb_4d2d_9bfe_2c9cba94ae24.slice/crio-34d694cdb95bc3a8bb99bb9acc23f48cbeeb991e02b7f85bb32ec942a86db503 WatchSource:0}: Error finding container 34d694cdb95bc3a8bb99bb9acc23f48cbeeb991e02b7f85bb32ec942a86db503: Status 404 returned error can't find the container with id 34d694cdb95bc3a8bb99bb9acc23f48cbeeb991e02b7f85bb32ec942a86db503
Sep 10 21:19:26.882862 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.882759    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3a5920_8efd_4b31_9aa5_03ff83238bd9.slice/crio-9faaf089d538c97c0b3a14ddafd44e592721ad4bdd629d9c9f61cedbcf9c1516 WatchSource:0}: Error finding container 9faaf089d538c97c0b3a14ddafd44e592721ad4bdd629d9c9f61cedbcf9c1516: Status 404 returned error can't find the container with id 9faaf089d538c97c0b3a14ddafd44e592721ad4bdd629d9c9f61cedbcf9c1516
Sep 10 21:19:26.883898 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.883760    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58d0cf93_576a_44e5_9fdc_3d7813d8e372.slice/crio-688c182697b054f0960bd0bf7584e40f7d9af7197976cdbc34322e1babbd2ec0 WatchSource:0}: Error finding container 688c182697b054f0960bd0bf7584e40f7d9af7197976cdbc34322e1babbd2ec0: Status 404 returned error can't find the container with id 688c182697b054f0960bd0bf7584e40f7d9af7197976cdbc34322e1babbd2ec0
Sep 10 21:19:26.885778 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.884999    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763c18c2_136b_41a3_b05d_4907bf940f7b.slice/crio-cbf905e6bb3fa9696e879e7ad504f5023ebb5429e1ef035d492871a261fa3e06 WatchSource:0}: Error finding container cbf905e6bb3fa9696e879e7ad504f5023ebb5429e1ef035d492871a261fa3e06: Status 404 returned error can't find the container with id cbf905e6bb3fa9696e879e7ad504f5023ebb5429e1ef035d492871a261fa3e06
Sep 10 21:19:26.885778 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.885567    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6388b71_4b19_4847_a45a_8c86d80b9bcd.slice/crio-ee19ec9ded9a50442d4f60820887b1f7fcd339b3d5cfc80a972e8117d5d95530 WatchSource:0}: Error finding container ee19ec9ded9a50442d4f60820887b1f7fcd339b3d5cfc80a972e8117d5d95530: Status 404 returned error can't find the container with id ee19ec9ded9a50442d4f60820887b1f7fcd339b3d5cfc80a972e8117d5d95530
Sep 10 21:19:26.886410 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:26.886390    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5bf0dc_7f92_4686_bd79_928113c62b59.slice/crio-334b47c97c44f53bd6ac028bce817fab1eb3b22d8f31bc44e10efff5992e62dd WatchSource:0}: Error finding container 334b47c97c44f53bd6ac028bce817fab1eb3b22d8f31bc44e10efff5992e62dd: Status 404 returned error can't find the container with id 334b47c97c44f53bd6ac028bce817fab1eb3b22d8f31bc44e10efff5992e62dd
Sep 10 21:19:26.963192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.963161    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bcwzb" event={"ID":"58d0cf93-576a-44e5-9fdc-3d7813d8e372","Type":"ContainerStarted","Data":"688c182697b054f0960bd0bf7584e40f7d9af7197976cdbc34322e1babbd2ec0"}
Sep 10 21:19:26.964507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.964481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8khf" event={"ID":"d00af7d9-fc18-43ef-b3ba-01279ef301bb","Type":"ContainerStarted","Data":"0733ad3f525318dc121a1b514fc36a69b359699b0d69b661e786ebeba1e115f4"}
Sep 10 21:19:26.965759 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.965733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn" event={"ID":"eae78f6f-4ca0-4651-bdef-b60eb4b84d21","Type":"ContainerStarted","Data":"77dee46917a7255d09a9b7e17df97c053d62819d04bcdc3fadeee9dd6ed919a0"}
Sep 10 21:19:26.967894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.967868    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"b2d6ebd71b368b317d437cef68944460","Type":"ContainerStarted","Data":"54a1d672b67706542254e1599966e797dde91d509f3ab1aae7f547ae1a651218"}
Sep 10 21:19:26.971570 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.971543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l69zp" event={"ID":"763c18c2-136b-41a3-b05d-4907bf940f7b","Type":"ContainerStarted","Data":"cbf905e6bb3fa9696e879e7ad504f5023ebb5429e1ef035d492871a261fa3e06"}
Sep 10 21:19:26.976130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.976102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerStarted","Data":"9faaf089d538c97c0b3a14ddafd44e592721ad4bdd629d9c9f61cedbcf9c1516"}
Sep 10 21:19:26.979435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.979413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"8564d82cfd765a9edb1199bf2b6cabd4b073186c7cf6c6e9ad046021e3bc70a5"}
Sep 10 21:19:26.981393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.981355    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" podStartSLOduration=1.9813432180000001 podStartE2EDuration="1.981343218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:26.981045107 +0000 UTC m=+3.475868647" watchObservedRunningTime="2026-09-10 21:19:26.981343218 +0000 UTC m=+3.476167007"
Sep 10 21:19:26.983472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.983433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5jfg" event={"ID":"5a5bf0dc-7f92-4686-bd79-928113c62b59","Type":"ContainerStarted","Data":"334b47c97c44f53bd6ac028bce817fab1eb3b22d8f31bc44e10efff5992e62dd"}
Sep 10 21:19:26.984650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.984626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ns59t" event={"ID":"a6388b71-4b19-4847-a45a-8c86d80b9bcd","Type":"ContainerStarted","Data":"ee19ec9ded9a50442d4f60820887b1f7fcd339b3d5cfc80a972e8117d5d95530"}
Sep 10 21:19:26.985806 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:26.985784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6w4fx" event={"ID":"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24","Type":"ContainerStarted","Data":"34d694cdb95bc3a8bb99bb9acc23f48cbeeb991e02b7f85bb32ec942a86db503"}
Sep 10 21:19:27.545991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.545667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:27.545991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.545738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:27.545991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.545787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:27.545991 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.545984    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:27.546348 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546007    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:27.546348 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546021    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:27.546348 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546087    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:29.546068263 +0000 UTC m=+6.040891788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:27.546907 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546521    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:27.546907 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546593    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:29.546565827 +0000 UTC m=+6.041389345 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:27.546907 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546661    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:27.546907 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.546698    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:29.546685487 +0000 UTC m=+6.041509012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:27.858509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.858374    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 21:14:26 +0000 UTC" deadline="2028-05-24 10:30:48.315314518 +0000 UTC"
Sep 10 21:19:27.858509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.858414    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14917h11m20.456904781s"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.952327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.952457    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.952937    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.953060    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.953134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:27.953326 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:27.953227    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:28.001037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:27.999070    2579 generic.go:356] "Generic (PLEG): container finished" podID="f1e34c96776044f47f6bc80f4249aad8" containerID="5574e9248616aa457c710eac7e370755652516089482e1385d98e7475685f565" exitCode=0
Sep 10 21:19:28.001037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:28.000099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"f1e34c96776044f47f6bc80f4249aad8","Type":"ContainerDied","Data":"5574e9248616aa457c710eac7e370755652516089482e1385d98e7475685f565"}
Sep 10 21:19:29.007720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.007003    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"f1e34c96776044f47f6bc80f4249aad8","Type":"ContainerStarted","Data":"69876f9dfb71f4f1889ded1c709e6ca04e6c62972ceebd1c98f52d974fc1248b"}
Sep 10 21:19:29.022662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.022610    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" podStartSLOduration=4.02258943 podStartE2EDuration="4.02258943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:29.021556271 +0000 UTC m=+5.516379810" watchObservedRunningTime="2026-09-10 21:19:29.02258943 +0000 UTC m=+5.517412970"
Sep 10 21:19:29.563044 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.563007    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:29.563213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.563100    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:29.563213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.563167    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:29.563213 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563198    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563216    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563224    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563241    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563243    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563294    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:33.563278198 +0000 UTC m=+10.058101731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563312    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:33.563302636 +0000 UTC m=+10.058126152 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:29.563384 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.563325    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:33.56331824 +0000 UTC m=+10.058141761 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:29.950865 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.950360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:29.950865 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.950490    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:29.951066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.950914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:29.951066 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.951009    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:29.951164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:29.951151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:29.951261 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:29.951241    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:31.951655 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:31.951628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:31.952022 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:31.951734    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:31.952077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:31.952028    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:31.952147 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:31.952126    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:31.952203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:31.952175    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:31.952251 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:31.952237    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:33.604935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.604684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.604963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.605002    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.604864    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605122    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:41.605064804 +0000 UTC m=+18.099888324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605137    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605152    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605168    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605180    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605201    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:41.605182146 +0000 UTC m=+18.100005666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:33.605403 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.605230    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:41.605218267 +0000 UTC m=+18.100041798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:33.951480 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.951405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:33.951635 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.951527    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:33.952174 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.951899    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:33.952174 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.952018    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:33.952174 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:33.952084    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:33.952174 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:33.952137    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:35.950559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:35.950527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:35.951013 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:35.950663    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:35.951196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:35.951167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:35.951311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:35.951277    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:35.951442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:35.951421    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:35.951541 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:35.951523    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:37.949995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:37.949963    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:37.949995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:37.949987    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:37.950424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:37.949974    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:37.950424 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:37.950103    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:37.950424 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:37.950206    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:37.950424 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:37.950298    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:39.950336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:39.950302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:39.950861 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:39.950302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:39.950861 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:39.950449    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:39.950861 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:39.950446    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:39.950861 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:39.950535    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:39.950861 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:39.950654    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:41.662905 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.662872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.662946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.662977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663038    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663059    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663071    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663074    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663122    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663130    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:57.663113503 +0000 UTC m=+34.157937034 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663204    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:19:57.663184865 +0000 UTC m=+34.158008388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:41.663311 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.663231    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:57.66321867 +0000 UTC m=+34.158042191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:41.949660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.949572    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:41.949660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.949606    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:41.949922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:41.949575    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:41.949922 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.949713    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:41.949922 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.949835    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:41.949922 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:41.949900    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:43.950500 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:43.950467    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:43.950965 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:43.950571    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:43.950965 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:43.950676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:43.950965 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:43.950811    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:43.950965 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:43.950898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:43.951177 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:43.950975    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:45.950070 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:45.949856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:45.950824 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:45.949856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:45.950824 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:45.950127    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:45.950824 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:45.949856    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:45.950824 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:45.950230    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:45.950824 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:45.950306    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:46.046505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.046464    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bcwzb" event={"ID":"58d0cf93-576a-44e5-9fdc-3d7813d8e372","Type":"ContainerStarted","Data":"ecd5a17172dd61908a6f247e173f38c6a79f9dc7830a0fcefc6398acde0f1a3c"}
Sep 10 21:19:46.048599 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.048573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h8khf" event={"ID":"d00af7d9-fc18-43ef-b3ba-01279ef301bb","Type":"ContainerStarted","Data":"4cc6bed2a4e6f7fa6c3598bb95a0e1cc7d03859996310e99368dd03f429f66a5"}
Sep 10 21:19:46.050228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.050204    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn" event={"ID":"eae78f6f-4ca0-4651-bdef-b60eb4b84d21","Type":"ContainerStarted","Data":"1a30a697c98daa9d98a787a09e1c86afd1c5b60f763d9f59855d6ac197cb298f"}
Sep 10 21:19:46.051681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.051660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l69zp" event={"ID":"763c18c2-136b-41a3-b05d-4907bf940f7b","Type":"ContainerStarted","Data":"675bc131a3f4d37273e1e55629ee69c87283d8b0b7cd9a760368e4649ba5b7a6"}
Sep 10 21:19:46.053256 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.053236    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="0223a7947f77f4c9101f595ccece593c9f9c8b24734a8e0b4d1860f7cf7274ce" exitCode=0
Sep 10 21:19:46.053353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.053299    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"0223a7947f77f4c9101f595ccece593c9f9c8b24734a8e0b4d1860f7cf7274ce"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056349    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"53d985dcc177ae37d35adff1f1bf78798e791110fc83e06751155a65738f4188"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056421    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"293b27dc2fce1880b36564ed3715bbf96299feb233a6278f8f69c6cc6994fb5d"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"103f3a7696a413f8ebc6c13c7b6f965f051711ccd94ae437d2ab89f733f60ebf"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"351090c0d7a8520550682268f4a9ff947e20e9d4ee2803acddafaa5174a99ca1"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056536    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"b1c036b717c4cea4a83201e2017ff5fa56bed8eeabc592155e28dc7b15eba2d6"}
Sep 10 21:19:46.056593 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.056575    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"0f4b55970d35eb68077d46f37ca61b3ed753a0e59482f9cb3d841e81b03ecbf6"}
Sep 10 21:19:46.057712 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.057681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ns59t" event={"ID":"a6388b71-4b19-4847-a45a-8c86d80b9bcd","Type":"ContainerStarted","Data":"699165c6b765074e5575ac6bd096f3ae52a901733a2fc2a3a357614dd89720fc"}
Sep 10 21:19:46.059097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.059075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6w4fx" event={"ID":"4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24","Type":"ContainerStarted","Data":"ed92f81d888a02f5f192c35d00c5b5b903ffe9d34dd4db5967ef5be9abd92d05"}
Sep 10 21:19:46.062129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.062093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bcwzb" podStartSLOduration=11.059065298 podStartE2EDuration="22.062081988s" totalImagesPullingTime="11.00301669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:46.061672795 +0000 UTC m=+22.556496332" watchObservedRunningTime="2026-09-10 21:19:46.062081988 +0000 UTC m=+22.556905527"
Sep 10 21:19:46.094606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.094574    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ns59t" podStartSLOduration=3.905219969 podStartE2EDuration="22.094562561s" totalImagesPullingTime="18.189342592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:46.075835094 +0000 UTC m=+22.570658632" watchObservedRunningTime="2026-09-10 21:19:46.094562561 +0000 UTC m=+22.589386099"
Sep 10 21:19:46.109575 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.109542    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l69zp" podStartSLOduration=3.919769614 podStartE2EDuration="22.109530104s" totalImagesPullingTime="18.18976049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:46.109189075 +0000 UTC m=+22.604012612" watchObservedRunningTime="2026-09-10 21:19:46.109530104 +0000 UTC m=+22.604353641"
Sep 10 21:19:46.139714 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.139672    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h8khf" podStartSLOduration=3.92688366 podStartE2EDuration="22.139660649s" totalImagesPullingTime="18.212776989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:46.139500713 +0000 UTC m=+22.634324250" watchObservedRunningTime="2026-09-10 21:19:46.139660649 +0000 UTC m=+22.634484186"
Sep 10 21:19:46.139882 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.139858    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6w4fx" podStartSLOduration=3.8818228059999997 podStartE2EDuration="22.13985215s" totalImagesPullingTime="18.258029344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:46.124652062 +0000 UTC m=+22.619475610" watchObservedRunningTime="2026-09-10 21:19:46.13985215 +0000 UTC m=+22.634675687"
Sep 10 21:19:46.170996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.170972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.235870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.235672    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.297413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.297390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:46.496918 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.496845    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 21:19:46.891281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.891188    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T21:19:46.496865412Z","UUID":"b377e2f3-72d7-41aa-a41d-3a83edc82d5d","Handler":null,"Name":"","Endpoint":""}
Sep 10 21:19:46.893012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.892982    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 21:19:46.893012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:46.893010    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 21:19:47.065252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.065214    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m5jfg" event={"ID":"5a5bf0dc-7f92-4686-bd79-928113c62b59","Type":"ContainerStarted","Data":"2fe941f29c3275a71937a8078962ddf4a94bafaeafe74f457a650990f2babead"}
Sep 10 21:19:47.067713 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.067645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn" event={"ID":"eae78f6f-4ca0-4651-bdef-b60eb4b84d21","Type":"ContainerStarted","Data":"182ccacf93e8b5d8a68ed56542c50b7391d3e06f2761784dd6e6e71ebdcaac2d"}
Sep 10 21:19:47.080215 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.080162    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m5jfg" podStartSLOduration=4.890385511 podStartE2EDuration="23.080145426s" totalImagesPullingTime="18.189759915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:47.080042643 +0000 UTC m=+23.574866183" watchObservedRunningTime="2026-09-10 21:19:47.080145426 +0000 UTC m=+23.574968965"
Sep 10 21:19:47.950475 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.950437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:47.950642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.950443    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:47.950642 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:47.950565    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:47.950642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:47.950574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:47.950857 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:47.950664    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:47.950857 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:47.950745    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:48.010907 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.010857    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:48.072918 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.072886    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn" event={"ID":"eae78f6f-4ca0-4651-bdef-b60eb4b84d21","Type":"ContainerStarted","Data":"b34549b304febab35dee20a2b1303db41b99dc4523f16b42736f862efac9e044"}
Sep 10 21:19:48.076802 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.076720    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"e7bf8fca83ea06e5bed4c359f9a56fa4df738e668405b0c688188f4551e6bd5b"}
Sep 10 21:19:48.094523 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.094483    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qjfkn" podStartSLOduration=3.738580314 podStartE2EDuration="24.094468709s" totalImagesPullingTime="20.355888395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:48.093982572 +0000 UTC m=+24.588806112" watchObservedRunningTime="2026-09-10 21:19:48.094468709 +0000 UTC m=+24.589292294"
Sep 10 21:19:48.400446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.400356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:48.500581 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:48.500542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.471410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.471358    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.525997 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.525967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.750528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.750456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.783629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.783598    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:49.784332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.784310    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:49.821116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.821089    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:49.950396 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.950364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:49.950562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.950364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:49.950562 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:49.950464    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:49.950682 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:49.950364    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:49.950682 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:49.950581    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:49.950766 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:49.950707    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:50.138611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:50.138552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:50.139266 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:50.139116    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bcwzb"
Sep 10 21:19:50.214725 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:50.214603    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.084421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.084387    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="bab390130ff005325b47a14f57f6404255f122aa8edc53df35361bc2ddda6b64" exitCode=0
Sep 10 21:19:51.084948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.084467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"bab390130ff005325b47a14f57f6404255f122aa8edc53df35361bc2ddda6b64"}
Sep 10 21:19:51.088622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.088597    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" event={"ID":"9c8523ff-06ac-4d0f-aa80-7646e3a74077","Type":"ContainerStarted","Data":"a99b23042f32eca9afc25b471259a8cd5bee4a04b76e240871f69e801f6901b4"}
Sep 10 21:19:51.132937 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.132900    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd" podStartSLOduration=8.879649191 podStartE2EDuration="27.132887251s" totalImagesPullingTime="18.25323806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:19:51.131869529 +0000 UTC m=+27.626693067" watchObservedRunningTime="2026-09-10 21:19:51.132887251 +0000 UTC m=+27.627710833"
Sep 10 21:19:51.170036 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.170005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.270866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.270842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:51.949744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.949713    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:51.949744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.949731    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:51.949974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:51.949710    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:51.949974 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:51.949860    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:51.949974 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:51.949892    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:51.949974 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:51.949966    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:52.003922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.003886    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k2tdn"]
Sep 10 21:19:52.007256 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.007228    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqrzz"]
Sep 10 21:19:52.007831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.007797    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kntrm"]
Sep 10 21:19:52.090418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.090384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:52.090418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.090384    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:52.091136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.090473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:52.091136 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:52.090596    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:52.091243 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:52.091111    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:52.091356 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:52.091329    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:52.091407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.091372    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:52.091447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.091420    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:52.091482 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.091448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:52.106689 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.106669    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:52.106900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:52.106883    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:19:53.094260 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:53.094228    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="736eec389b0735025f7b43f6b4169e3742366d8e51f52f4213b146f4266214a4" exitCode=0
Sep 10 21:19:53.094715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:53.094306    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"736eec389b0735025f7b43f6b4169e3742366d8e51f52f4213b146f4266214a4"}
Sep 10 21:19:53.951407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:53.951374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:53.951407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:53.951408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:53.951583 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:53.951503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:53.951583 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:53.951549    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:53.951647 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:53.951583    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:53.951647 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:53.951615    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:55.104011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.103917    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="32acb25a68f4510a99c0dff02cad7ca4d200edb42fdbc51d38485316721fae0c" exitCode=0
Sep 10 21:19:55.104011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.103955    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"32acb25a68f4510a99c0dff02cad7ca4d200edb42fdbc51d38485316721fae0c"}
Sep 10 21:19:55.343824 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.343795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.443653 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.443585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.754017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.753927    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.809575 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.809547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:55.949691 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.949654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:55.949691 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.949672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:55.949987 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:55.949663    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:55.949987 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:55.949819    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kntrm" podUID="f72cb332-6751-46d0-885a-7b29a340c898"
Sep 10 21:19:55.949987 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:55.949885    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xqrzz" podUID="62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5"
Sep 10 21:19:55.949987 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:55.949959    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k2tdn" podUID="3f478cf5-0402-452a-8a3d-981ed11e30cf"
Sep 10 21:19:56.642650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:56.642612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:56.731912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:56.731855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.088832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.088790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.149459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.149426    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.358348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.358240    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.471148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.471120    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.542013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.541981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.603611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.603578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.680883 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.680852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.680908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.680932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.680975    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.680998    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681010    2579 projected.go:196] Error preparing data for projected volume kube-api-access-m5vp5 for pod openshift-network-diagnostics/network-check-target-kntrm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681022    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681058    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681061    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5 podName:f72cb332-6751-46d0-885a-7b29a340c898 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:29.681046651 +0000 UTC m=+66.175870190 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m5vp5" (UniqueName: "kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5") pod "network-check-target-kntrm" (UID: "f72cb332-6751-46d0-885a-7b29a340c898") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681139    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs podName:3f478cf5-0402-452a-8a3d-981ed11e30cf nodeName:}" failed. No retries permitted until 2026-09-10 21:20:29.681118174 +0000 UTC m=+66.175941692 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs") pod "network-metrics-daemon-k2tdn" (UID: "3f478cf5-0402-452a-8a3d-981ed11e30cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 21:19:57.681214 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.681163    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret podName:62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:29.681151739 +0000 UTC m=+66.175975258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret") pod "global-pull-secret-syncer-xqrzz" (UID: "62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5") : object "kube-system"/"original-pull-secret" not registered
Sep 10 21:19:57.758082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.758055    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeReady"
Sep 10 21:19:57.758253 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.758195    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 21:19:57.803138 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.803103    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jc6p7"]
Sep 10 21:19:57.806880 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.806857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.809164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.809139    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 21:19:57.809286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.809166    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 21:19:57.809286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.809182    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvtg"]
Sep 10 21:19:57.809286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.809241    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f6g5g\""
Sep 10 21:19:57.812353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.812326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:57.814437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.814420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 21:19:57.814562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.814464    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bgqsb\""
Sep 10 21:19:57.814562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.814420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 21:19:57.814562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.814425    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 21:19:57.816982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.816472    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jc6p7"]
Sep 10 21:19:57.825166 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.825145    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvtg"]
Sep 10 21:19:57.829341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.829324    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2g8rg"]
Sep 10 21:19:57.832708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.832688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.835413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.835391    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 21:19:57.835511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.835436    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 21:19:57.835563 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.835395    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 21:19:57.835625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.835436    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 21:19:57.835814 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.835770    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jf9vp\""
Sep 10 21:19:57.844452 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.844430    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2g8rg"]
Sep 10 21:19:57.882391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.882313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9vcj\" (UniqueName: \"kubernetes.io/projected/3ac182e9-57a2-4739-a9f2-d16d19831c1e-kube-api-access-c9vcj\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.882391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.882362    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.882591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.882422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ac182e9-57a2-4739-a9f2-d16d19831c1e-config-volume\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.882591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.882446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac182e9-57a2-4739-a9f2-d16d19831c1e-tmp-dir\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.950462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.950430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:19:57.950627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.950462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:19:57.950627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.950547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:19:57.953049 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8fbc\""
Sep 10 21:19:57.953348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953162    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kc896\""
Sep 10 21:19:57.953348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953204    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:19:57.953348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953237    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:19:57.953711 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953515    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:19:57.953711 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.953554    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:19:57.963033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.963010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:57.983479 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs4wk\" (UniqueName: \"kubernetes.io/projected/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-api-access-cs4wk\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.983591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ac182e9-57a2-4739-a9f2-d16d19831c1e-config-volume\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.983591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac182e9-57a2-4739-a9f2-d16d19831c1e-tmp-dir\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.983591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983545    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-data-volume\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.983591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0467875-9d2b-485c-9177-b9e075d8dce5-cert\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:57.983799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.983799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983657    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-crio-socket\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.983799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9vcj\" (UniqueName: \"kubernetes.io/projected/3ac182e9-57a2-4739-a9f2-d16d19831c1e-kube-api-access-c9vcj\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.983799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ac182e9-57a2-4739-a9f2-d16d19831c1e-tmp-dir\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.983978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:57.983978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.983978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.983917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zzrg\" (UniqueName: \"kubernetes.io/projected/e0467875-9d2b-485c-9177-b9e075d8dce5-kube-api-access-5zzrg\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:57.984075 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.984010    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:19:57.984075 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:57.984056    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls podName:3ac182e9-57a2-4739-a9f2-d16d19831c1e nodeName:}" failed. No retries permitted until 2026-09-10 21:19:58.484042806 +0000 UTC m=+34.978866323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls") pod "dns-default-jc6p7" (UID: "3ac182e9-57a2-4739-a9f2-d16d19831c1e") : secret "dns-default-metrics-tls" not found
Sep 10 21:19:57.984172 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.984093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3ac182e9-57a2-4739-a9f2-d16d19831c1e-config-volume\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:57.995137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:57.995007    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9vcj\" (UniqueName: \"kubernetes.io/projected/3ac182e9-57a2-4739-a9f2-d16d19831c1e-kube-api-access-c9vcj\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:58.038942 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.038916    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:58.084930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.084904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zzrg\" (UniqueName: \"kubernetes.io/projected/e0467875-9d2b-485c-9177-b9e075d8dce5-kube-api-access-5zzrg\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:58.085071 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.084953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs4wk\" (UniqueName: \"kubernetes.io/projected/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-api-access-cs4wk\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-data-volume\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0467875-9d2b-485c-9177-b9e075d8dce5-cert\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:58.085313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-crio-socket\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085515 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085446    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-data-volume\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.085568 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.085534    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:19:58.085568 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.085546    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:58.085663 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.085610    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:58.585592162 +0000 UTC m=+35.080415679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:58.085663 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.085644    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:58.585631819 +0000 UTC m=+35.080455335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : configmap "kube-rbac-proxy" not found
Sep 10 21:19:58.085808 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.085725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-crio-socket\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.088106 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.088089    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0467875-9d2b-485c-9177-b9e075d8dce5-cert\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:58.095532 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.095508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs4wk\" (UniqueName: \"kubernetes.io/projected/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-api-access-cs4wk\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.095821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.095764    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zzrg\" (UniqueName: \"kubernetes.io/projected/e0467875-9d2b-485c-9177-b9e075d8dce5-kube-api-access-5zzrg\") pod \"ingress-canary-gcvtg\" (UID: \"e0467875-9d2b-485c-9177-b9e075d8dce5\") " pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:58.126964 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.126923    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gcvtg"
Sep 10 21:19:58.487767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.487724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:58.487970 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.487906    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:19:58.488037 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.487981    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls podName:3ac182e9-57a2-4739-a9f2-d16d19831c1e nodeName:}" failed. No retries permitted until 2026-09-10 21:19:59.487964831 +0000 UTC m=+35.982788365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls") pod "dns-default-jc6p7" (UID: "3ac182e9-57a2-4739-a9f2-d16d19831c1e") : secret "dns-default-metrics-tls" not found
Sep 10 21:19:58.588632 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.588594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.588817 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.588721    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:19:58.588817 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.588723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:58.588817 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.588801    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:59.588765645 +0000 UTC m=+36.083589161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : configmap "kube-rbac-proxy" not found
Sep 10 21:19:58.588968 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.588819    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:58.588968 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:58.588865    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:19:59.588848566 +0000 UTC m=+36.083672082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:58.639874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.639848    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l69zp_763c18c2-136b-41a3-b05d-4907bf940f7b/dns-node-resolver/0.log"
Sep 10 21:19:58.752110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.752016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:58.836701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.836664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:58.956089 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.956059    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-nlkqw"]
Sep 10 21:19:58.959455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.959430    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:58.961718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.961695    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 21:19:58.961874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.961703    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m25l5\""
Sep 10 21:19:58.961874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.961704    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 21:19:58.962487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.962469    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 21:19:58.962591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.962489    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 21:19:58.967503 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.967063    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-nlkqw"]
Sep 10 21:19:58.996859 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:58.996819    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:59.071917 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.071852    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:19:59.093918 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.093889    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ldwr\" (UniqueName: \"kubernetes.io/projected/26efe7a0-0cdf-4701-9c2c-2621717100a5-kube-api-access-5ldwr\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.094070 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.093970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-cabundle\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.094070 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.094029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-key\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.195226 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.195172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ldwr\" (UniqueName: \"kubernetes.io/projected/26efe7a0-0cdf-4701-9c2c-2621717100a5-kube-api-access-5ldwr\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.195371 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.195253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-cabundle\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.195371 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.195290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-key\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.195915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.195891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-cabundle\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.198183 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.198154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/26efe7a0-0cdf-4701-9c2c-2621717100a5-signing-key\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.203625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.203589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ldwr\" (UniqueName: \"kubernetes.io/projected/26efe7a0-0cdf-4701-9c2c-2621717100a5-kube-api-access-5ldwr\") pod \"service-ca-568869fdc5-nlkqw\" (UID: \"26efe7a0-0cdf-4701-9c2c-2621717100a5\") " pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.271032 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.271004    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-nlkqw"
Sep 10 21:19:59.454651 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.454625    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gcvtg"]
Sep 10 21:19:59.498288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.498249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:19:59.498406 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.498362    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:19:59.498444 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.498421    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls podName:3ac182e9-57a2-4739-a9f2-d16d19831c1e nodeName:}" failed. No retries permitted until 2026-09-10 21:20:01.498401319 +0000 UTC m=+37.993224834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls") pod "dns-default-jc6p7" (UID: "3ac182e9-57a2-4739-a9f2-d16d19831c1e") : secret "dns-default-metrics-tls" not found
Sep 10 21:19:59.542671 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.542647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-nlkqw"]
Sep 10 21:19:59.545946 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:59.545913    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26efe7a0_0cdf_4701_9c2c_2621717100a5.slice/crio-c4a8f79f161ac8abfeceea08aa96a6cff2ec960d4a1c137335043e487fad3097 WatchSource:0}: Error finding container c4a8f79f161ac8abfeceea08aa96a6cff2ec960d4a1c137335043e487fad3097: Status 404 returned error can't find the container with id c4a8f79f161ac8abfeceea08aa96a6cff2ec960d4a1c137335043e487fad3097
Sep 10 21:19:59.546420 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:19:59.546377    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0467875_9d2b_485c_9177_b9e075d8dce5.slice/crio-a0565868053336c49d637fcd63bc3325b8105aa3297bbaad8fe703d47fe8b38d WatchSource:0}: Error finding container a0565868053336c49d637fcd63bc3325b8105aa3297bbaad8fe703d47fe8b38d: Status 404 returned error can't find the container with id a0565868053336c49d637fcd63bc3325b8105aa3297bbaad8fe703d47fe8b38d
Sep 10 21:19:59.599427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.599406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:59.599508 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.599474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:19:59.599551 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.599537    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 21:19:59.599598 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.599570    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:59.599598 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.599596    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:01.599582217 +0000 UTC m=+38.094405733 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : configmap "kube-rbac-proxy" not found
Sep 10 21:19:59.599676 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:19:59.599612    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:01.599601271 +0000 UTC m=+38.094424787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:19:59.639454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:19:59.639432    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ns59t_a6388b71-4b19-4847-a45a-8c86d80b9bcd/node-ca/0.log"
Sep 10 21:20:00.118965 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:00.118659    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-nlkqw" event={"ID":"26efe7a0-0cdf-4701-9c2c-2621717100a5","Type":"ContainerStarted","Data":"c4a8f79f161ac8abfeceea08aa96a6cff2ec960d4a1c137335043e487fad3097"}
Sep 10 21:20:00.122898 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:00.122867    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="18ecc06f5bb2380ef8f2687ad5cc43fc00d049c136b7620cf245eb4da7478a7c" exitCode=0
Sep 10 21:20:00.123037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:00.122964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"18ecc06f5bb2380ef8f2687ad5cc43fc00d049c136b7620cf245eb4da7478a7c"}
Sep 10 21:20:00.125738 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:00.125704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcvtg" event={"ID":"e0467875-9d2b-485c-9177-b9e075d8dce5","Type":"ContainerStarted","Data":"a0565868053336c49d637fcd63bc3325b8105aa3297bbaad8fe703d47fe8b38d"}
Sep 10 21:20:00.229127 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:00.229096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.072978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.072917    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.132012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.131980    2579 generic.go:356] "Generic (PLEG): container finished" podID="fb3a5920-8efd-4b31-9aa5-03ff83238bd9" containerID="78ddfdb17ea807a6e17a3788eac97a0f8aaa90107ad77e1b794b0d1da638484a" exitCode=0
Sep 10 21:20:01.132452 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.132012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerDied","Data":"78ddfdb17ea807a6e17a3788eac97a0f8aaa90107ad77e1b794b0d1da638484a"}
Sep 10 21:20:01.138103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.138043    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.237569 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.237531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:01.517118 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.517090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:01.517251 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:01.517204    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:20:01.517293 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:01.517262    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls podName:3ac182e9-57a2-4739-a9f2-d16d19831c1e nodeName:}" failed. No retries permitted until 2026-09-10 21:20:05.517247294 +0000 UTC m=+42.012070832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls") pod "dns-default-jc6p7" (UID: "3ac182e9-57a2-4739-a9f2-d16d19831c1e") : secret "dns-default-metrics-tls" not found
Sep 10 21:20:01.617687 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.617641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:01.617868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.617749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:01.617868 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:01.617814    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:20:01.617978 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:01.617890    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:05.617868804 +0000 UTC m=+42.112692335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:20:01.625176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:01.625148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:02.347552 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:02.347519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:02.448795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:02.448754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:03.027429 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.027396    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:03.138445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.138412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-nlkqw" event={"ID":"26efe7a0-0cdf-4701-9c2c-2621717100a5","Type":"ContainerStarted","Data":"b3173af38afc5401a3715369f48327dcb20e61bc81d7c268f8256a01a05d63bf"}
Sep 10 21:20:03.141929 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.141900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" event={"ID":"fb3a5920-8efd-4b31-9aa5-03ff83238bd9","Type":"ContainerStarted","Data":"1237f97e8b09e0d81ed63bc35786a7878c9645c187039d10c8d63b81a40505fb"}
Sep 10 21:20:03.143316 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.143294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gcvtg" event={"ID":"e0467875-9d2b-485c-9177-b9e075d8dce5","Type":"ContainerStarted","Data":"1b0b5832465f07af97008aee1fca8540e2ee94d45743746dd48faf46bed2beea"}
Sep 10 21:20:03.153987 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.153943    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-nlkqw" podStartSLOduration=2.240758549 podStartE2EDuration="5.153930179s" totalImagesPullingTime="2.91317163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:03.153176982 +0000 UTC m=+39.648000523" watchObservedRunningTime="2026-09-10 21:20:03.153930179 +0000 UTC m=+39.648753716"
Sep 10 21:20:03.177482 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.177425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nkwbq" podStartSLOduration=10.86856097 podStartE2EDuration="39.177409756s" totalImagesPullingTime="28.308848786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:03.176625162 +0000 UTC m=+39.671448700" watchObservedRunningTime="2026-09-10 21:20:03.177409756 +0000 UTC m=+39.672233295"
Sep 10 21:20:03.190786 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.190748    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gcvtg" podStartSLOduration=3.272528604 podStartE2EDuration="6.19073744s" totalImagesPullingTime="2.918208836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:03.189960626 +0000 UTC m=+39.684784164" watchObservedRunningTime="2026-09-10 21:20:03.19073744 +0000 UTC m=+39.685560977"
Sep 10 21:20:03.911545 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:03.911517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.024093 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.024037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.149504 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.149468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.408085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.408055    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.503938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.503904    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.639137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.639104    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:04.705995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:04.705893    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:05.199610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:05.199578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:05.550356 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:05.550319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:05.550516 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:05.550477    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 21:20:05.550577 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:05.550565    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls podName:3ac182e9-57a2-4739-a9f2-d16d19831c1e nodeName:}" failed. No retries permitted until 2026-09-10 21:20:13.550547029 +0000 UTC m=+50.045370545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls") pod "dns-default-jc6p7" (UID: "3ac182e9-57a2-4739-a9f2-d16d19831c1e") : secret "dns-default-metrics-tls" not found
Sep 10 21:20:05.651247 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:05.651213    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:05.651414 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:05.651384    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 21:20:05.651475 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:20:05.651458    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls podName:10b3bcec-62b2-40ab-a79a-cc6923ed2b61 nodeName:}" failed. No retries permitted until 2026-09-10 21:20:13.651437002 +0000 UTC m=+50.146260540 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2g8rg" (UID: "10b3bcec-62b2-40ab-a79a-cc6923ed2b61") : secret "insights-runtime-extractor-tls" not found
Sep 10 21:20:05.795608 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:05.795574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.162551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:06.162517    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.263949 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:06.263923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.407265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:06.407239    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.602075 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:06.602048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:06.996177 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:06.996098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:07.201827 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:07.201766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:07.594048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:07.593985    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:07.648462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:07.648428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:08.230395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:08.230363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:08.321007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:08.320981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:08.808890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:08.808856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.010298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:09.010267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.400607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:09.400575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.602299 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:09.602267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:09.806590 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:09.806559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:10.222821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:10.222714    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:10.999287 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:10.999257    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:11.198323 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:11.198289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:11.399984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:11.399950    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:11.504421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:11.504387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:11.605426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:11.605397    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:11.803546 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:11.803515    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:13.028341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.028309    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:13.199917 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.199880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:13.399565 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.399483    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:13.606168 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.606128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:13.613320 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.613290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:13.616440 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.616416    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ac182e9-57a2-4739-a9f2-d16d19831c1e-metrics-tls\") pod \"dns-default-jc6p7\" (UID: \"3ac182e9-57a2-4739-a9f2-d16d19831c1e\") " pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:13.714485 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.714411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:13.716669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.716652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/10b3bcec-62b2-40ab-a79a-cc6923ed2b61-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2g8rg\" (UID: \"10b3bcec-62b2-40ab-a79a-cc6923ed2b61\") " pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:13.719583 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.719558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:13.741875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.741854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2g8rg"
Sep 10 21:20:13.800529 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:13.800490    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:14.005411 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.005349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jc6p7"]
Sep 10 21:20:14.008162 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:14.008127    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ac182e9_57a2_4739_a9f2_d16d19831c1e.slice/crio-cd755f4075c4c5f4bdb8dcef642bc6dd395cb4f241706cfe93e2cc9ac5c261cd WatchSource:0}: Error finding container cd755f4075c4c5f4bdb8dcef642bc6dd395cb4f241706cfe93e2cc9ac5c261cd: Status 404 returned error can't find the container with id cd755f4075c4c5f4bdb8dcef642bc6dd395cb4f241706cfe93e2cc9ac5c261cd
Sep 10 21:20:14.026636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.026614    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2g8rg"]
Sep 10 21:20:14.029140 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:14.029118    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b3bcec_62b2_40ab_a79a_cc6923ed2b61.slice/crio-1a3b57196a620657eae19ad52216ba44b6083f451f8706af1515cafc50d5d565 WatchSource:0}: Error finding container 1a3b57196a620657eae19ad52216ba44b6083f451f8706af1515cafc50d5d565: Status 404 returned error can't find the container with id 1a3b57196a620657eae19ad52216ba44b6083f451f8706af1515cafc50d5d565
Sep 10 21:20:14.173344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.173305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2g8rg" event={"ID":"10b3bcec-62b2-40ab-a79a-cc6923ed2b61","Type":"ContainerStarted","Data":"d54382acbd3c331999d184fda1f9f7509cde3f2f50a36ad2502ff4d2d6d64099"}
Sep 10 21:20:14.173486 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.173375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2g8rg" event={"ID":"10b3bcec-62b2-40ab-a79a-cc6923ed2b61","Type":"ContainerStarted","Data":"1a3b57196a620657eae19ad52216ba44b6083f451f8706af1515cafc50d5d565"}
Sep 10 21:20:14.174502 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.174472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jc6p7" event={"ID":"3ac182e9-57a2-4739-a9f2-d16d19831c1e","Type":"ContainerStarted","Data":"cd755f4075c4c5f4bdb8dcef642bc6dd395cb4f241706cfe93e2cc9ac5c261cd"}
Sep 10 21:20:14.414409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.414322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:14.603303 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:14.603180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:16.180568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:16.180539    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2g8rg" event={"ID":"10b3bcec-62b2-40ab-a79a-cc6923ed2b61","Type":"ContainerStarted","Data":"82c14b0b7248b73e144af45bfe1e376bd9bfb4916fafe79563dac837ecae246f"}
Sep 10 21:20:17.186002 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:17.185976    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jc6p7" event={"ID":"3ac182e9-57a2-4739-a9f2-d16d19831c1e","Type":"ContainerStarted","Data":"d037e76dfcaa259a461ea99259cb4d82199b037c074698257019b9320ca47378"}
Sep 10 21:20:17.186363 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:17.186070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jc6p7" event={"ID":"3ac182e9-57a2-4739-a9f2-d16d19831c1e","Type":"ContainerStarted","Data":"a68f6279f4c9c601eec4112717b27f4424ed247d8807255d30037be03313f398"}
Sep 10 21:20:17.186363 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:17.186113    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:17.204519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:17.204477    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jc6p7" podStartSLOduration=18.110780021 podStartE2EDuration="20.204462441s" totalImagesPullingTime="2.09368242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:17.203214288 +0000 UTC m=+53.698037826" watchObservedRunningTime="2026-09-10 21:20:17.204462441 +0000 UTC m=+53.699285979"
Sep 10 21:20:18.193842 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:18.193807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2g8rg" event={"ID":"10b3bcec-62b2-40ab-a79a-cc6923ed2b61","Type":"ContainerStarted","Data":"9ada21d43a2f726ff9f0c301f41d640a26934f836374084482b89060c82b5b54"}
Sep 10 21:20:18.212437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:18.212398    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2g8rg" podStartSLOduration=18.301938825 podStartE2EDuration="21.212383742s" totalImagesPullingTime="2.910444917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:18.211745913 +0000 UTC m=+54.706569453" watchObservedRunningTime="2026-09-10 21:20:18.212383742 +0000 UTC m=+54.707207280"
Sep 10 21:20:18.749638 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:18.749609    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:18.815695 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:18.815668    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.015419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.015384    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"]
Sep 10 21:20:20.018457 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.018435    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"]
Sep 10 21:20:20.018579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.018563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.021008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.020987    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"]
Sep 10 21:20:20.021283 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.021255    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.021877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.021857    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 21:20:20.022112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.022092    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 10 21:20:20.022112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.022103    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 21:20:20.022511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.022494    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 21:20:20.023555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.023535    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 10 21:20:20.023886 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.023562    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xf8pm\""
Sep 10 21:20:20.024750 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.024732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.026874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.026813    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 21:20:20.026874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.026839    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 21:20:20.026874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.026862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 21:20:20.027051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.026910    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 21:20:20.036358 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.036329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"]
Sep 10 21:20:20.037526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.037507    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"]
Sep 10 21:20:20.053142 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.053117    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"]
Sep 10 21:20:20.061855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.061836    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbnz9\" (UniqueName: \"kubernetes.io/projected/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-kube-api-access-pbnz9\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.061959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.061868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lkr\" (UniqueName: \"kubernetes.io/projected/63040c20-7d08-47d1-ae66-57fa6054ec78-kube-api-access-d4lkr\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.061959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.061896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggpk7\" (UniqueName: \"kubernetes.io/projected/34f7b5c8-5455-4edb-9e62-48aff06775a2-kube-api-access-ggpk7\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.062072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.061957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63040c20-7d08-47d1-ae66-57fa6054ec78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.062072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.062072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062039    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.062173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.062173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-tmp\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.062173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062128    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.062289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.062289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.062194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34f7b5c8-5455-4edb-9e62-48aff06775a2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.162536 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lkr\" (UniqueName: \"kubernetes.io/projected/63040c20-7d08-47d1-ae66-57fa6054ec78-kube-api-access-d4lkr\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggpk7\" (UniqueName: \"kubernetes.io/projected/34f7b5c8-5455-4edb-9e62-48aff06775a2-kube-api-access-ggpk7\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63040c20-7d08-47d1-ae66-57fa6054ec78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-tmp\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.162716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.163085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.163085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34f7b5c8-5455-4edb-9e62-48aff06775a2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.163085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.162897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbnz9\" (UniqueName: \"kubernetes.io/projected/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-kube-api-access-pbnz9\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.163235 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.163155    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-tmp\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.163803 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.163703    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/63040c20-7d08-47d1-ae66-57fa6054ec78-additional-service-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.165722 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.165623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-ca\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.166056 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.165872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.166129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.166094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.166129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.166103    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.166491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.166470    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/34f7b5c8-5455-4edb-9e62-48aff06775a2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.166801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.166762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/63040c20-7d08-47d1-ae66-57fa6054ec78-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.177267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.177210    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbnz9\" (UniqueName: \"kubernetes.io/projected/67b4f2c7-3dac-4c7e-9433-c40de86f4b0b-kube-api-access-pbnz9\") pod \"klusterlet-addon-workmgr-5f6b695b9d-8lg5f\" (UID: \"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.177948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.177929    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggpk7\" (UniqueName: \"kubernetes.io/projected/34f7b5c8-5455-4edb-9e62-48aff06775a2-kube-api-access-ggpk7\") pod \"managed-serviceaccount-addon-agent-7fffcf8868-6npz2\" (UID: \"34f7b5c8-5455-4edb-9e62-48aff06775a2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.178851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.178833    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lkr\" (UniqueName: \"kubernetes.io/projected/63040c20-7d08-47d1-ae66-57fa6054ec78-kube-api-access-d4lkr\") pod \"cluster-proxy-proxy-agent-744c859d5-c8ph6\" (UID: \"63040c20-7d08-47d1-ae66-57fa6054ec78\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.196564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.196540    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.281229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.281171    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.330513 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.330486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:20.344186 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.344161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"
Sep 10 21:20:20.348995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.348964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"
Sep 10 21:20:20.393040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.392635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.483947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.483910    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.581831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.581719    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:20.606996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.606962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"]
Sep 10 21:20:20.609969 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:20.609942    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67b4f2c7_3dac_4c7e_9433_c40de86f4b0b.slice/crio-c48df4e8ca1021fc84951d3ceb7610becba06379084282d9cb40205635a0925e WatchSource:0}: Error finding container c48df4e8ca1021fc84951d3ceb7610becba06379084282d9cb40205635a0925e: Status 404 returned error can't find the container with id c48df4e8ca1021fc84951d3ceb7610becba06379084282d9cb40205635a0925e
Sep 10 21:20:20.617629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.617598    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2"]
Sep 10 21:20:20.620158 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:20.620135    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34f7b5c8_5455_4edb_9e62_48aff06775a2.slice/crio-7e697c73b476a38e1d30f53cda41f942d2ed6aa8a9f9fe5a2d36a559b6cfd89c WatchSource:0}: Error finding container 7e697c73b476a38e1d30f53cda41f942d2ed6aa8a9f9fe5a2d36a559b6cfd89c: Status 404 returned error can't find the container with id 7e697c73b476a38e1d30f53cda41f942d2ed6aa8a9f9fe5a2d36a559b6cfd89c
Sep 10 21:20:20.639562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:20.639540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6"]
Sep 10 21:20:20.643436 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:20.643415    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63040c20_7d08_47d1_ae66_57fa6054ec78.slice/crio-72bd138813b8de779c251df4fbdfd714ad463218b7840d770bc7c2400cdcca10 WatchSource:0}: Error finding container 72bd138813b8de779c251df4fbdfd714ad463218b7840d770bc7c2400cdcca10: Status 404 returned error can't find the container with id 72bd138813b8de779c251df4fbdfd714ad463218b7840d770bc7c2400cdcca10
Sep 10 21:20:21.166527 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.166492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:21.204044 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.204009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2" event={"ID":"34f7b5c8-5455-4edb-9e62-48aff06775a2","Type":"ContainerStarted","Data":"7e697c73b476a38e1d30f53cda41f942d2ed6aa8a9f9fe5a2d36a559b6cfd89c"}
Sep 10 21:20:21.205266 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.205237    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6" event={"ID":"63040c20-7d08-47d1-ae66-57fa6054ec78","Type":"ContainerStarted","Data":"72bd138813b8de779c251df4fbdfd714ad463218b7840d770bc7c2400cdcca10"}
Sep 10 21:20:21.206341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.206321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f" event={"ID":"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b","Type":"ContainerStarted","Data":"c48df4e8ca1021fc84951d3ceb7610becba06379084282d9cb40205635a0925e"}
Sep 10 21:20:21.268496 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.268468    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:21.457412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.457331    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:21.538329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:21.538216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.464966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.464899    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.576835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.574026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.664966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.664925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.751159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.751080    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.883737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.883498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:22.982057 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:22.982029    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:23.314940 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:23.314907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:23.420135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:23.420106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:23.707176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:23.707108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:23.766941 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:23.766908    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.105939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.105887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.110686 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.110658    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kv2bd"
Sep 10 21:20:24.206872 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.206834    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.312011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.311974    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.409044 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.408959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:24.650378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:24.650339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:25.735413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:25.735375    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:25.795724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:25.795678    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:26.670946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:26.670906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:26.757663 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:26.757629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:27.196417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:27.196389    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jc6p7"
Sep 10 21:20:27.242294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:27.242092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:28.235886 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.235848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2" event={"ID":"34f7b5c8-5455-4edb-9e62-48aff06775a2","Type":"ContainerStarted","Data":"01479112628b29acd3d2a8c1b5be573cf7050c890a9e84fad1d57f3de3dc9e4e"}
Sep 10 21:20:28.238255 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.238230    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6" event={"ID":"63040c20-7d08-47d1-ae66-57fa6054ec78","Type":"ContainerStarted","Data":"6de2f9a624ef7504ca95c0abcf18e6092cc3e4fc800c2a945db5acebbe1e8a2c"}
Sep 10 21:20:28.238369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.238269    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6" event={"ID":"63040c20-7d08-47d1-ae66-57fa6054ec78","Type":"ContainerStarted","Data":"cc6767a4148ef1e6ffa8e55cd6eb650488e181affaa64cb1d4a86a0dedca8f11"}
Sep 10 21:20:28.238369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.238289    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6" event={"ID":"63040c20-7d08-47d1-ae66-57fa6054ec78","Type":"ContainerStarted","Data":"a8987cc7db9eb782052f74d7ba2bbb50d126c60d42fcaee658489bd40e3a5b8e"}
Sep 10 21:20:28.239759 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.239736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f" event={"ID":"67b4f2c7-3dac-4c7e-9433-c40de86f4b0b","Type":"ContainerStarted","Data":"1bb30145fba82f5b1e5e41294dfdf6e4a52b35636cbad010f346856f9bc37ba4"}
Sep 10 21:20:28.240033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.240015    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:28.241463 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.241440    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f"
Sep 10 21:20:28.251841 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.251799    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fffcf8868-6npz2" podStartSLOduration=2.65830959 podStartE2EDuration="9.251767616s" totalImagesPullingTime="6.593458026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:28.251325823 +0000 UTC m=+64.746149361" watchObservedRunningTime="2026-09-10 21:20:28.251767616 +0000 UTC m=+64.746591153"
Sep 10 21:20:28.270330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.270291    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-744c859d5-c8ph6" podStartSLOduration=2.675781409 podStartE2EDuration="9.270276526s" totalImagesPullingTime="6.594495117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:28.269509422 +0000 UTC m=+64.764332961" watchObservedRunningTime="2026-09-10 21:20:28.270276526 +0000 UTC m=+64.765100056"
Sep 10 21:20:28.288000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.286284    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6b695b9d-8lg5f" podStartSLOduration=2.657373034 podStartE2EDuration="9.286263891s" totalImagesPullingTime="6.628890857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:28.285144743 +0000 UTC m=+64.779968276" watchObservedRunningTime="2026-09-10 21:20:28.286263891 +0000 UTC m=+64.781087432"
Sep 10 21:20:28.521169 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.521144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:28.622207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.622180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:28.686197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:28.686174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:29.737564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.737524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:20:29.738057 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.737588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:20:29.738057 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.737616    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:20:29.740050 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.740025    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 21:20:29.740137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.740069    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 21:20:29.740428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.740414    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 21:20:29.749568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.749546    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 21:20:29.750106 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.750087    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5-original-pull-secret\") pod \"global-pull-secret-syncer-xqrzz\" (UID: \"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5\") " pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:20:29.750179 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.750148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f478cf5-0402-452a-8a3d-981ed11e30cf-metrics-certs\") pod \"network-metrics-daemon-k2tdn\" (UID: \"3f478cf5-0402-452a-8a3d-981ed11e30cf\") " pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:20:29.760519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.760498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5vp5\" (UniqueName: \"kubernetes.io/projected/f72cb332-6751-46d0-885a-7b29a340c898-kube-api-access-m5vp5\") pod \"network-check-target-kntrm\" (UID: \"f72cb332-6751-46d0-885a-7b29a340c898\") " pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:20:29.763426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.763405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xqrzz"
Sep 10 21:20:29.771592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.771575    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kc896\""
Sep 10 21:20:29.777405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.777391    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-c8fbc\""
Sep 10 21:20:29.780019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.780000    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:20:29.786352 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:29.786340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k2tdn"
Sep 10 21:20:30.065577 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.065548    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xqrzz"]
Sep 10 21:20:30.069055 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:30.069025    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62848a85_c65b_4a2d_95f0_c7ebaa3a4fc5.slice/crio-bdf389b98f85a2a56eb7c900fb168b06e1ee601c55dabb23c72d086b7d48043c WatchSource:0}: Error finding container bdf389b98f85a2a56eb7c900fb168b06e1ee601c55dabb23c72d086b7d48043c: Status 404 returned error can't find the container with id bdf389b98f85a2a56eb7c900fb168b06e1ee601c55dabb23c72d086b7d48043c
Sep 10 21:20:30.108466 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.108445    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kntrm"]
Sep 10 21:20:30.111084 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:30.111065    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72cb332_6751_46d0_885a_7b29a340c898.slice/crio-65817dc5aaf2db964ea751dc68503e93ba3e8e7ab458999c5a68fcce71ed0377 WatchSource:0}: Error finding container 65817dc5aaf2db964ea751dc68503e93ba3e8e7ab458999c5a68fcce71ed0377: Status 404 returned error can't find the container with id 65817dc5aaf2db964ea751dc68503e93ba3e8e7ab458999c5a68fcce71ed0377
Sep 10 21:20:30.139615 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.139592    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k2tdn"]
Sep 10 21:20:30.142571 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:30.142550    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f478cf5_0402_452a_8a3d_981ed11e30cf.slice/crio-3ce43aa6e06b1195aef1e5f3e61af5abbfe699df2dff58dc5479d874cb596cc3 WatchSource:0}: Error finding container 3ce43aa6e06b1195aef1e5f3e61af5abbfe699df2dff58dc5479d874cb596cc3: Status 404 returned error can't find the container with id 3ce43aa6e06b1195aef1e5f3e61af5abbfe699df2dff58dc5479d874cb596cc3
Sep 10 21:20:30.246607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.246573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqrzz" event={"ID":"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5","Type":"ContainerStarted","Data":"bdf389b98f85a2a56eb7c900fb168b06e1ee601c55dabb23c72d086b7d48043c"}
Sep 10 21:20:30.247850 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.247823    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2tdn" event={"ID":"3f478cf5-0402-452a-8a3d-981ed11e30cf","Type":"ContainerStarted","Data":"3ce43aa6e06b1195aef1e5f3e61af5abbfe699df2dff58dc5479d874cb596cc3"}
Sep 10 21:20:30.248938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.248914    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kntrm" event={"ID":"f72cb332-6751-46d0-885a-7b29a340c898","Type":"ContainerStarted","Data":"65817dc5aaf2db964ea751dc68503e93ba3e8e7ab458999c5a68fcce71ed0377"}
Sep 10 21:20:30.369183 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:30.369105    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:31.290903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:31.290864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:31.391558 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:31.391508    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:31.453104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:31.453073    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:32.260115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.260034    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2tdn" event={"ID":"3f478cf5-0402-452a-8a3d-981ed11e30cf","Type":"ContainerStarted","Data":"cdccea7ad2b0ba659405cdf45a982ca6f9f7fa9d2ef5cf2884c47bf2a42f834d"}
Sep 10 21:20:32.260115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.260101    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k2tdn" event={"ID":"3f478cf5-0402-452a-8a3d-981ed11e30cf","Type":"ContainerStarted","Data":"f7926eb07d77530db6b6bdba7850c9e1f3be308c591b2b7dba91d6d9085a6728"}
Sep 10 21:20:32.277912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.277511    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k2tdn" podStartSLOduration=66.717764903 podStartE2EDuration="1m8.277489504s" totalImagesPullingTime="1.559724601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:32.275887911 +0000 UTC m=+68.770711448" watchObservedRunningTime="2026-09-10 21:20:32.277489504 +0000 UTC m=+68.772313044"
Sep 10 21:20:32.379133 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.379096    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:32.440611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.440578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:32.853567 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.853528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:32.943968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:32.943932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.052425 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.052389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.118160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.118082    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m75sz"]
Sep 10 21:20:33.139946 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.139922    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.141591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.141560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.142706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.142574    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 21:20:33.142706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.142581    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zzvmr\""
Sep 10 21:20:33.143077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.143057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 21:20:33.143171 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.143076    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 21:20:33.143235 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.143062    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 21:20:33.143879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.143516    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 21:20:33.143879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.143718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 21:20:33.266674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.266645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.270762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270729    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-tls\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.270903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270810    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-sys\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.270903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-metrics-client-ca\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.270903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-accelerators-collector-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.271088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-textfile\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.271088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkzx5\" (UniqueName: \"kubernetes.io/projected/b6a7c330-a164-4b38-8639-ce4eeadd7f17-kube-api-access-qkzx5\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.271088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-wtmp\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.271088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.270986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-root\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.271088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.271011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.368639 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.368563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.372052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372023    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-root\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-tls\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-sys\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-metrics-client-ca\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372218    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-accelerators-collector-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-textfile\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkzx5\" (UniqueName: \"kubernetes.io/projected/b6a7c330-a164-4b38-8639-ce4eeadd7f17-kube-api-access-qkzx5\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-wtmp\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372452    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-wtmp\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372498    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-root\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.372959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.372937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b6a7c330-a164-4b38-8639-ce4eeadd7f17-sys\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.373300 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.373269    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-textfile\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.373553 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.373530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-metrics-client-ca\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.373617 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.373586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-accelerators-collector-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.375794 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.375752    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-tls\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.376088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.376066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b6a7c330-a164-4b38-8639-ce4eeadd7f17-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.385152 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.385111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkzx5\" (UniqueName: \"kubernetes.io/projected/b6a7c330-a164-4b38-8639-ce4eeadd7f17-kube-api-access-qkzx5\") pod \"node-exporter-m75sz\" (UID: \"b6a7c330-a164-4b38-8639-ce4eeadd7f17\") " pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.452760 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.452726    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m75sz"
Sep 10 21:20:33.472110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.472061    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.569719 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.569673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:33.668739 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:33.668636    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.076378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:34.076343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.186947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:34.186905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.338156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:34.338039    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.441829 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:34.441766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.954336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:34.954308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:34.995995 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:20:34.995900    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a7c330_a164_4b38_8639_ce4eeadd7f17.slice/crio-88f4d20803aa385870947bd9dbba4cd5f3f0e4cc475efca6ef6b94181b2b209f WatchSource:0}: Error finding container 88f4d20803aa385870947bd9dbba4cd5f3f0e4cc475efca6ef6b94181b2b209f: Status 404 returned error can't find the container with id 88f4d20803aa385870947bd9dbba4cd5f3f0e4cc475efca6ef6b94181b2b209f
Sep 10 21:20:35.019329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.019308    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:35.276154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.276114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m75sz" event={"ID":"b6a7c330-a164-4b38-8639-ce4eeadd7f17","Type":"ContainerStarted","Data":"88f4d20803aa385870947bd9dbba4cd5f3f0e4cc475efca6ef6b94181b2b209f"}
Sep 10 21:20:35.277703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.277675    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kntrm" event={"ID":"f72cb332-6751-46d0-885a-7b29a340c898","Type":"ContainerStarted","Data":"18567d49e97399caf987c4e05eacef0353d1e3f202f51b3cc4dc86b3b6384ae3"}
Sep 10 21:20:35.277846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.277807    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:20:35.279285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.279263    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xqrzz" event={"ID":"62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5","Type":"ContainerStarted","Data":"b3296b9310b531ef8414ad26591ad4a3ab6109ccb542bab7a80307bbd7b90eae"}
Sep 10 21:20:35.292506 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.292462    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kntrm" podStartSLOduration=66.410092077 podStartE2EDuration="1m11.292446538s" totalImagesPullingTime="4.882354461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:35.29200449 +0000 UTC m=+71.786828027" watchObservedRunningTime="2026-09-10 21:20:35.292446538 +0000 UTC m=+71.787270077"
Sep 10 21:20:35.305072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:35.305034    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xqrzz" podStartSLOduration=66.368553809 podStartE2EDuration="1m11.305021242s" totalImagesPullingTime="4.936467433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:35.304443699 +0000 UTC m=+71.799267238" watchObservedRunningTime="2026-09-10 21:20:35.305021242 +0000 UTC m=+71.799844815"
Sep 10 21:20:36.038283 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.038243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:36.139203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.139174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:36.195628 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.195570    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:36.275915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.275884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:36.284640 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.284616    2579 generic.go:356] "Generic (PLEG): container finished" podID="b6a7c330-a164-4b38-8639-ce4eeadd7f17" containerID="0f1eab943b3cb62c0b0e380723b3d4ad899f8108071925cd174d406c532d227e" exitCode=0
Sep 10 21:20:36.284756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.284706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m75sz" event={"ID":"b6a7c330-a164-4b38-8639-ce4eeadd7f17","Type":"ContainerDied","Data":"0f1eab943b3cb62c0b0e380723b3d4ad899f8108071925cd174d406c532d227e"}
Sep 10 21:20:36.372302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:36.372278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:37.290141 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:37.290105    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m75sz" event={"ID":"b6a7c330-a164-4b38-8639-ce4eeadd7f17","Type":"ContainerStarted","Data":"97550e0d5ad262ef2257efc5f394a378b4b9c0dd370d4d12d22749fc0574646d"}
Sep 10 21:20:37.290601 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:37.290155    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m75sz" event={"ID":"b6a7c330-a164-4b38-8639-ce4eeadd7f17","Type":"ContainerStarted","Data":"71cd5e95bd8fdf7d26811f2353db134a18e6b62e94a0d3954bdf98b3e4985aa0"}
Sep 10 21:20:37.307665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:37.307603    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m75sz" podStartSLOduration=3.308290721 podStartE2EDuration="4.307586475s" totalImagesPullingTime="931.05824ms" totalInitContainerRuntime="68.237514ms" isStatefulPod=true podCreationTimestamp="2026-09-10 21:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:20:37.307492375 +0000 UTC m=+73.802315915" watchObservedRunningTime="2026-09-10 21:20:37.307586475 +0000 UTC m=+73.802410011"
Sep 10 21:20:37.852321 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:37.852290    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:37.950283 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:37.950251    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:38.545377 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:38.545339    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:38.633650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:38.633618    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:38.886381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:38.886297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:38.963004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:38.962973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:39.292568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:39.292537    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:42.259365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:42.259330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:42.358153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:42.358124    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.134681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:43.134649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.230930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:43.230895    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:43.296423 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:43.296389    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:45.483097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:45.482981    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:45.543156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:45.543125    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:45.603473 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:45.603437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:48.158173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:48.158135    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:48.240648 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:48.240620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.192674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:51.192637    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.292876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:51.292849    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.547615 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:51.547571    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:51.648967 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:51.648935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:54.568998 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:54.568963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:54.667746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:54.667711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:59.335573 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:59.335539    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:20:59.396793 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:20:59.396757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:06.158956 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:06.158925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:06.262801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:06.262742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:06.286731 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:06.286706    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kntrm"
Sep 10 21:21:21.165259 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:21.165218    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:21.266039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:21.266009    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:35.060508 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:35.060472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:52.281238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:52.281202    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:21:52.482054 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:21:52.482021    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:00.344364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:00.344329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:08.141020 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:08.140979    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:11.385306 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:11.385276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:11.486897 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:11.486869    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:11.551919 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:11.551891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:47.987004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:47.986973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:57.278795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:57.278748    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:57.926764 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:57.926733    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:22:58.028149 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:22:58.028115    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:07.518617 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:07.518574    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:07.627382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:07.627343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.556949 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:16.556896    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:16.622752 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:16.622723    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.540670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:19.540640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:19.641557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:19.641533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.705614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:27.705582    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.789624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:27.789596    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:27.860628 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:27.860606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:36.180026 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:36.179987    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:36.274821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:36.274770    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:57.379115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:57.379024    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:23:57.486966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:23:57.486935    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:19.418095 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:19.418054    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:20.889655 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:20.889623    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:23.274343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:23.274306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:23.374720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:23.374693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:23.832026 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:23.831994    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 21:24:24.658614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:24.658587    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:24.757112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:24.757082    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:25.209277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:25.209244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:25.342608 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:25.342527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:26.658968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:26.658925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:27.047219 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:27.047185    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.030422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:30.030371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:30.177491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:30.177456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:35.656546 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:35.656472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:36.263357 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:36.263310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:36.320372 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:36.320328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:36.438715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:36.438671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:36.536705 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:36.536673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:41.659249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:41.659216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:41.730092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:41.730065    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:43.778367 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:43.778338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:57.704670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:57.704641    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:57.807733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:57.807696    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:58.299953 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.299920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:58.637382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.637281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:58.742109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.742078    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:58.762340 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.762309    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"]
Sep 10 21:24:58.765332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.765318    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.768348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.768331    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 21:24:58.768449 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.768331    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 21:24:58.768497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.768445    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jn67t\""
Sep 10 21:24:58.768671 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.768659    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 21:24:58.768864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.768848    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 21:24:58.781531 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.781512    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"]
Sep 10 21:24:58.814702 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.814670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvm7q\" (UniqueName: \"kubernetes.io/projected/5200c0c5-382f-4333-9890-325309b0cd2f-kube-api-access-tvm7q\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.814702 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.814718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-manifests\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.814932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.814738    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-tmp\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.814932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.814863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.814932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.814908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-manifests\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-tmp\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvm7q\" (UniqueName: \"kubernetes.io/projected/5200c0c5-382f-4333-9890-325309b0cd2f-kube-api-access-tvm7q\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.915989 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.915965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-manifests\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.918102 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.918074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5200c0c5-382f-4333-9890-325309b0cd2f-tmp\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.918284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.918268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.918326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.918268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5200c0c5-382f-4333-9890-325309b0cd2f-webhook-cert\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:58.925953 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.925926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:58.926300 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:58.926284    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvm7q\" (UniqueName: \"kubernetes.io/projected/5200c0c5-382f-4333-9890-325309b0cd2f-kube-api-access-tvm7q\") pod \"opendatahub-operator-controller-manager-56fc979fdc-sjwrs\" (UID: \"5200c0c5-382f-4333-9890-325309b0cd2f\") " pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:59.004263 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.004233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:59.071947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.071918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:59.074892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.074875    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:24:59.177104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.177034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:59.280679 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.280651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:24:59.353628 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.353599    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"]
Sep 10 21:24:59.356736 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:24:59.356710    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5200c0c5_382f_4333_9890_325309b0cd2f.slice/crio-d4f59b23e33fa88de85ee7141cda84d8264486693814b71260e607464cf4db92 WatchSource:0}: Error finding container d4f59b23e33fa88de85ee7141cda84d8264486693814b71260e607464cf4db92: Status 404 returned error can't find the container with id d4f59b23e33fa88de85ee7141cda84d8264486693814b71260e607464cf4db92
Sep 10 21:24:59.358378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.358360    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:24:59.372616 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:24:59.372585    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:00.134733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:00.134691    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs" event={"ID":"5200c0c5-382f-4333-9890-325309b0cd2f","Type":"ContainerStarted","Data":"d4f59b23e33fa88de85ee7141cda84d8264486693814b71260e607464cf4db92"}
Sep 10 21:25:03.147059 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:03.147023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs" event={"ID":"5200c0c5-382f-4333-9890-325309b0cd2f","Type":"ContainerStarted","Data":"0a3430ad60f4a2c810b8dba04151bbfcbfabba56cda6cb15d832fe9055beaa83"}
Sep 10 21:25:04.151681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:04.151640    2579 generic.go:356] "Generic (PLEG): container finished" podID="5200c0c5-382f-4333-9890-325309b0cd2f" containerID="0a3430ad60f4a2c810b8dba04151bbfcbfabba56cda6cb15d832fe9055beaa83" exitCode=0
Sep 10 21:25:04.152178 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:04.151701    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs" event={"ID":"5200c0c5-382f-4333-9890-325309b0cd2f","Type":"ContainerDied","Data":"0a3430ad60f4a2c810b8dba04151bbfcbfabba56cda6cb15d832fe9055beaa83"}
Sep 10 21:25:05.157037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:05.157002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs" event={"ID":"5200c0c5-382f-4333-9890-325309b0cd2f","Type":"ContainerStarted","Data":"662aaa2fc898e86f93f0cc121a1c99a54bc8eff6c691bc9bee06f17507091c15"}
Sep 10 21:25:05.157460 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:05.157143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:25:05.180023 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:05.179970    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs" podStartSLOduration=2.879646022 podStartE2EDuration="7.179957459s" totalImagesPullingTime="3.308727067s" totalInitContainerRuntime="991.58437ms" isStatefulPod=false podCreationTimestamp="2026-09-10 21:24:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:05.178654407 +0000 UTC m=+341.673477946" watchObservedRunningTime="2026-09-10 21:25:05.179957459 +0000 UTC m=+341.674781074"
Sep 10 21:25:10.035795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:10.035742    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:10.130047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:10.130005    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:16.162055 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.161973    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56fc979fdc-sjwrs"
Sep 10 21:25:16.270409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.270378    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"]
Sep 10 21:25:16.273283 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.273268    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.275361 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.275339    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 10 21:25:16.275361 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.275355    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 21:25:16.275654 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.275410    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-m54tm\""
Sep 10 21:25:16.275767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.275660    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 10 21:25:16.275767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.275741    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 10 21:25:16.276100 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.276084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 21:25:16.282770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.282750    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"]
Sep 10 21:25:16.336015 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.335983    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.336161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.336021    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrc49\" (UniqueName: \"kubernetes.io/projected/dba3ebc2-c15b-444a-a228-97725058b983-kube-api-access-rrc49\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.336161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.336055    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dba3ebc2-c15b-444a-a228-97725058b983-manager-config\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.336262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.336171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-metrics-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.437375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.437308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-metrics-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.437375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.437336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.437375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.437359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrc49\" (UniqueName: \"kubernetes.io/projected/dba3ebc2-c15b-444a-a228-97725058b983-kube-api-access-rrc49\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.437616 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.437380    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dba3ebc2-c15b-444a-a228-97725058b983-manager-config\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.437985 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.437965    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/dba3ebc2-c15b-444a-a228-97725058b983-manager-config\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.439978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.439959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-metrics-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.440061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.440040    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dba3ebc2-c15b-444a-a228-97725058b983-cert\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.446119 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.446099    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrc49\" (UniqueName: \"kubernetes.io/projected/dba3ebc2-c15b-444a-a228-97725058b983-kube-api-access-rrc49\") pod \"lws-controller-manager-5644d67b8d-dqfkx\" (UID: \"dba3ebc2-c15b-444a-a228-97725058b983\") " pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.583101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.583064    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:16.856041 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:16.856014    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"]
Sep 10 21:25:16.859487 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:25:16.859454    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba3ebc2_c15b_444a_a228_97725058b983.slice/crio-5f696496a13159a62c433c4f406f8bbbf631c4c97ad764755bce5363a541fd8c WatchSource:0}: Error finding container 5f696496a13159a62c433c4f406f8bbbf631c4c97ad764755bce5363a541fd8c: Status 404 returned error can't find the container with id 5f696496a13159a62c433c4f406f8bbbf631c4c97ad764755bce5363a541fd8c
Sep 10 21:25:17.199656 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:17.199574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx" event={"ID":"dba3ebc2-c15b-444a-a228-97725058b983","Type":"ContainerStarted","Data":"5f696496a13159a62c433c4f406f8bbbf631c4c97ad764755bce5363a541fd8c"}
Sep 10 21:25:18.018101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:18.018037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:18.172114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:18.172077    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:18.550003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:18.549970    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:18.940054 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:18.939969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:20.195858 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:20.195810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:20.212759 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:20.212732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx" event={"ID":"dba3ebc2-c15b-444a-a228-97725058b983","Type":"ContainerStarted","Data":"3efd73d025316a85808ba54cb025443beead21803d8de77a79c5ae2dd3d05eab"}
Sep 10 21:25:20.212910 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:20.212890    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:20.259434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:20.259387    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx" podStartSLOduration=1.837046902 podStartE2EDuration="4.259371389s" totalImagesPullingTime="2.422324487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:20.258192669 +0000 UTC m=+356.753016207" watchObservedRunningTime="2026-09-10 21:25:20.259371389 +0000 UTC m=+356.754194947"
Sep 10 21:25:20.332994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:20.332967    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:21.401273 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:21.401236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:25.178434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.178401    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:25.272206 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.272177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:25.416590 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.416559    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"]
Sep 10 21:25:25.421153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.421135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.425695 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.425670    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9x5j5\""
Sep 10 21:25:25.426203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.426184    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 21:25:25.426558 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.426537    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 21:25:25.427004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.426983    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 21:25:25.460852 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.460766    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"]
Sep 10 21:25:25.495135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.495091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:25.502308 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502446    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpj7m\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-kube-api-access-xpj7m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2842bf44-3f01-49e1-b186-c151e13b0778-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2842bf44-3f01-49e1-b186-c151e13b0778-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502679 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.502679 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.502649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603367 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603335    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2842bf44-3f01-49e1-b186-c151e13b0778-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603436    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpj7m\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-kube-api-access-xpj7m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603943 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603551    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2842bf44-3f01-49e1-b186-c151e13b0778-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603943 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.603943 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.603902    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.604297 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.604253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2842bf44-3f01-49e1-b186-c151e13b0778-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.604405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.604385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.604470 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.604437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.605876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.605842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2842bf44-3f01-49e1-b186-c151e13b0778-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.606071 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.606054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2842bf44-3f01-49e1-b186-c151e13b0778-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.614160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.614132    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.614160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.614149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpj7m\" (UniqueName: \"kubernetes.io/projected/2842bf44-3f01-49e1-b186-c151e13b0778-kube-api-access-xpj7m\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs\" (UID: \"2842bf44-3f01-49e1-b186-c151e13b0778\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.692302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.692278    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:25.731467 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.731408    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:25.867935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:25.867902    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:26.004234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:26.004211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"]
Sep 10 21:25:26.006435 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:25:26.006411    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2842bf44_3f01_49e1_b186_c151e13b0778.slice/crio-f1bd323362c7b27a6adc92156ac8951863565067197c15419586e38763fba846 WatchSource:0}: Error finding container f1bd323362c7b27a6adc92156ac8951863565067197c15419586e38763fba846: Status 404 returned error can't find the container with id f1bd323362c7b27a6adc92156ac8951863565067197c15419586e38763fba846
Sep 10 21:25:26.233445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:26.233409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs" event={"ID":"2842bf44-3f01-49e1-b186-c151e13b0778","Type":"ContainerStarted","Data":"f1bd323362c7b27a6adc92156ac8951863565067197c15419586e38763fba846"}
Sep 10 21:25:29.021197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.021156    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:25:29.021476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.021235    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:25:29.021476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.021265    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:25:29.150390 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.150353    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfmvp"]
Sep 10 21:25:29.153402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.153386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.155661 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.155637    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-b8m2d\""
Sep 10 21:25:29.155769 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.155636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 10 21:25:29.162487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.162461    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfmvp"]
Sep 10 21:25:29.212594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.212559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.232391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.232355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn4g9\" (UniqueName: \"kubernetes.io/projected/06462169-a019-4ace-a50b-0ee178d060c4-kube-api-access-hn4g9\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.232563 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.232406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.247053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.247017    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs" event={"ID":"2842bf44-3f01-49e1-b186-c151e13b0778","Type":"ContainerStarted","Data":"8674a7cd9c2fd651ad7f359bc463266be98bd77ee474f8944b8b1d7a0314f7c3"}
Sep 10 21:25:29.274395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.274306    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs" podStartSLOduration=1.2618219019999999 podStartE2EDuration="4.274290401s" totalImagesPullingTime="3.012468499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:29.27289926 +0000 UTC m=+365.767722800" watchObservedRunningTime="2026-09-10 21:25:29.274290401 +0000 UTC m=+365.769113939"
Sep 10 21:25:29.312876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.312847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.333912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.333883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn4g9\" (UniqueName: \"kubernetes.io/projected/06462169-a019-4ace-a50b-0ee178d060c4-kube-api-access-hn4g9\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.333912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.333922    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.334136 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:25:29.334076    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 10 21:25:29.334194 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:25:29.334162    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert podName:06462169-a019-4ace-a50b-0ee178d060c4 nodeName:}" failed. No retries permitted until 2026-09-10 21:25:29.834135944 +0000 UTC m=+366.328959472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert") pod "odh-model-controller-7998c67dc4-dfmvp" (UID: "06462169-a019-4ace-a50b-0ee178d060c4") : secret "odh-model-controller-webhook-cert" not found
Sep 10 21:25:29.344864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.344838    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn4g9\" (UniqueName: \"kubernetes.io/projected/06462169-a019-4ace-a50b-0ee178d060c4-kube-api-access-hn4g9\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.413629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.413595    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.513889 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.513859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.727499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.727431    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vngxn"]
Sep 10 21:25:29.730417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.730401    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:29.731859 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.731841    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:29.733500 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.733482    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 21:25:29.744631 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.744611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vngxn"]
Sep 10 21:25:29.755462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.755438    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:29.837051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.837025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpdx\" (UniqueName: \"kubernetes.io/projected/dd012855-932e-4c18-a81d-d2162ee5610f-kube-api-access-jzpdx\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:29.837207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.837077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.837207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.837159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:29.839553 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.839536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06462169-a019-4ace-a50b-0ee178d060c4-cert\") pod \"odh-model-controller-7998c67dc4-dfmvp\" (UID: \"06462169-a019-4ace-a50b-0ee178d060c4\") " pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:29.938383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.938358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:29.938531 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.938394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpdx\" (UniqueName: \"kubernetes.io/projected/dd012855-932e-4c18-a81d-d2162ee5610f-kube-api-access-jzpdx\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:29.938581 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:25:29.938529    2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 10 21:25:29.938613 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:25:29.938603    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs podName:dd012855-932e-4c18-a81d-d2162ee5610f nodeName:}" failed. No retries permitted until 2026-09-10 21:25:30.438583849 +0000 UTC m=+366.933407371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs") pod "model-serving-api-54d89bd5fc-vngxn" (UID: "dd012855-932e-4c18-a81d-d2162ee5610f") : secret "model-serving-api-tls" not found
Sep 10 21:25:29.946868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:29.946840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpdx\" (UniqueName: \"kubernetes.io/projected/dd012855-932e-4c18-a81d-d2162ee5610f-kube-api-access-jzpdx\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:30.064680 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.064654    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:30.180302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.180268    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:30.332350 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.332329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dfmvp"]
Sep 10 21:25:30.334846 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:25:30.334819    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06462169_a019_4ace_a50b_0ee178d060c4.slice/crio-983d6cc950e7d9dd3ffbaf5bf73fd3ab3718fb186d564a15ff9d311c942e4288 WatchSource:0}: Error finding container 983d6cc950e7d9dd3ffbaf5bf73fd3ab3718fb186d564a15ff9d311c942e4288: Status 404 returned error can't find the container with id 983d6cc950e7d9dd3ffbaf5bf73fd3ab3718fb186d564a15ff9d311c942e4288
Sep 10 21:25:30.443268 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.443238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:30.445669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.445653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd012855-932e-4c18-a81d-d2162ee5610f-tls-certs\") pod \"model-serving-api-54d89bd5fc-vngxn\" (UID: \"dd012855-932e-4c18-a81d-d2162ee5610f\") " pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:30.641115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.641031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:30.736357 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.736333    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:30.909394 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:30.909368    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vngxn"]
Sep 10 21:25:30.911466 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:25:30.911441    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd012855_932e_4c18_a81d_d2162ee5610f.slice/crio-4b697f21a22d28362a675d5f4de7ab8f9bbaecfae0ddb7c61336a115ec6282c8 WatchSource:0}: Error finding container 4b697f21a22d28362a675d5f4de7ab8f9bbaecfae0ddb7c61336a115ec6282c8: Status 404 returned error can't find the container with id 4b697f21a22d28362a675d5f4de7ab8f9bbaecfae0ddb7c61336a115ec6282c8
Sep 10 21:25:31.218577 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:31.218500    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5644d67b8d-dqfkx"
Sep 10 21:25:31.257254 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:31.257014    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp" event={"ID":"06462169-a019-4ace-a50b-0ee178d060c4","Type":"ContainerStarted","Data":"983d6cc950e7d9dd3ffbaf5bf73fd3ab3718fb186d564a15ff9d311c942e4288"}
Sep 10 21:25:31.260430 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:31.260400    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn" event={"ID":"dd012855-932e-4c18-a81d-d2162ee5610f","Type":"ContainerStarted","Data":"4b697f21a22d28362a675d5f4de7ab8f9bbaecfae0ddb7c61336a115ec6282c8"}
Sep 10 21:25:31.260662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:31.260639    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:31.262039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:31.262017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs"
Sep 10 21:25:32.687003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:32.686972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:35.277492 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.277456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn" event={"ID":"dd012855-932e-4c18-a81d-d2162ee5610f","Type":"ContainerStarted","Data":"1da475cf3186e9a98719f7dcef04e07703d1ab1288cbd6b57e78f612578f8156"}
Sep 10 21:25:35.277947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.277610    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:25:35.279007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.278987    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp" event={"ID":"06462169-a019-4ace-a50b-0ee178d060c4","Type":"ContainerStarted","Data":"a90fd5e50afcf6a036329c41438daee50259644a01d1ff4da038c3d7dc84cbb6"}
Sep 10 21:25:35.279125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.279112    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:35.306022 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.305964    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn" podStartSLOduration=2.719751675 podStartE2EDuration="6.305945875s" totalImagesPullingTime="3.5861942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:35.305368748 +0000 UTC m=+371.800192286" watchObservedRunningTime="2026-09-10 21:25:35.305945875 +0000 UTC m=+371.800769406"
Sep 10 21:25:35.333253 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:35.333209    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp" podStartSLOduration=2.206270669 podStartE2EDuration="6.333192346s" totalImagesPullingTime="4.126921677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:35.331549867 +0000 UTC m=+371.826373401" watchObservedRunningTime="2026-09-10 21:25:35.333192346 +0000 UTC m=+371.828015919"
Sep 10 21:25:36.037094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.037052    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"]
Sep 10 21:25:36.041113 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.040714    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.044161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.044130    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 21:25:36.044288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.044189    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 21:25:36.044383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.044369    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-58hrr\""
Sep 10 21:25:36.044444 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.044387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 21:25:36.054877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.054854    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"]
Sep 10 21:25:36.189127 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189127 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfsj7\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-kube-api-access-qfsj7\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189151    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c71e42-581a-4c01-9c95-12fc34c48b71-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.189327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.189320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.290589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.290589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.290589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfsj7\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-kube-api-access-qfsj7\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.291139 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.291139 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290639    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.291139 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.291139 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.290837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c71e42-581a-4c01-9c95-12fc34c48b71-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.291599 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.291573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.293226 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.293200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.293341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.293312    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.293398 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.293343    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/32c71e42-581a-4c01-9c95-12fc34c48b71-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.293451 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.293413    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.301183 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.301163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.302890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.302870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfsj7\" (UniqueName: \"kubernetes.io/projected/32c71e42-581a-4c01-9c95-12fc34c48b71-kube-api-access-qfsj7\") pod \"istiod-openshift-gateway-5677ccb49b-7q5k9\" (UID: \"32c71e42-581a-4c01-9c95-12fc34c48b71\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.349977 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.349957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:36.629570 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:36.629502    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"]
Sep 10 21:25:36.632024 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:25:36.631980    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c71e42_581a_4c01_9c95_12fc34c48b71.slice/crio-e37709b53346204432b65305209064b108966316728e76c131d77b4e9ca79d42 WatchSource:0}: Error finding container e37709b53346204432b65305209064b108966316728e76c131d77b4e9ca79d42: Status 404 returned error can't find the container with id e37709b53346204432b65305209064b108966316728e76c131d77b4e9ca79d42
Sep 10 21:25:37.287656 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:37.287616    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9" event={"ID":"32c71e42-581a-4c01-9c95-12fc34c48b71","Type":"ContainerStarted","Data":"e37709b53346204432b65305209064b108966316728e76c131d77b4e9ca79d42"}
Sep 10 21:25:39.269953 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:39.269911    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:25:39.270224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:39.269984    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:25:39.426267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:39.426235    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:39.545869 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:39.545766    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:39.923431 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:39.923300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:40.075191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.075157    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:40.301690 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.301656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9" event={"ID":"32c71e42-581a-4c01-9c95-12fc34c48b71","Type":"ContainerStarted","Data":"792aabc46c2a1fe2c49cc270231509c0958761e00c86a656ec6dd36581c46c26"}
Sep 10 21:25:40.302092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.301723    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:40.303455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.303423    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-7q5k9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:25:40.303620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.303477    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9" podUID="32c71e42-581a-4c01-9c95-12fc34c48b71" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:25:40.715641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.715565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:40.785607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.785573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:40.970648 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:40.970566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:41.305578 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:41.305555    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9"
Sep 10 21:25:41.362185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:41.362141    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7q5k9" podStartSLOduration=2.726629716 podStartE2EDuration="5.362126334s" totalImagesPullingTime="2.635496618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:25:40.370472409 +0000 UTC m=+376.865295947" watchObservedRunningTime="2026-09-10 21:25:41.362126334 +0000 UTC m=+377.856949871"
Sep 10 21:25:43.599130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:43.599101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:44.031176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:44.031146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:25:46.285207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:46.285178    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dfmvp"
Sep 10 21:25:46.287770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:25:46.287748    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vngxn"
Sep 10 21:26:12.320627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:12.320588    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:12.432230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:12.432195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:14.367747 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:14.367711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:14.468836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:14.468769    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:30.138639 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:30.138600    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:30.235399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:30.235367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:43.840143 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:43.840114    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:46.636407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:46.636322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:48.779352 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:48.779312    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:49.034380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:49.034281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:49.329509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:49.329417    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:49.687052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:49.686968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:50.003040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:50.002954    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:50.780309 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:50.780269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:50.980582 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:50.980549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:52.883850 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:52.883811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:53.250112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:53.249999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:54.474501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:54.474466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:54.577500 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:54.577466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:55.995350 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:55.995313    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:56.093651 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:56.093616    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:56.819144 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:56.819098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:56.915286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:56.915261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:59.794398 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:59.794362    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:26:59.919483 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:26:59.919446    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:00.406137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:00.406090    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:00.507591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:00.507527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.407262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:04.407215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.508184 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:04.508138    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:04.607955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:04.607919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:13.276549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:13.276513    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:13.373492 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:13.373458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:16.059385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:16.059343    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:16.163293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:16.163253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:19.121721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:19.121676    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:19.229913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:19.229880    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:22.638245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:22.638189    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:22.709352 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:22.709323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:23.641094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:23.641060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:23.740321 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:23.740287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:23.813647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:23.813614    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:32.459483 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:32.459410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:32.563668 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:32.563613    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:40.332066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:40.332030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:41.022585 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:41.022551    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.002197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:45.002166    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.107100 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:45.107058    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.207861 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:45.207826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.839405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:45.839364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:27:45.906527 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:27:45.906493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.165052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:01.165020    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.267668 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:01.267633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:01.648187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:01.648158    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:07.244505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:07.244466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:11.535530 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:11.535479    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:12.213978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:12.213949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:13.347308 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.347277    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:28:13.350344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.350326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.353604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.353582    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j8k9d\""
Sep 10 21:28:13.353604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.353599    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 21:28:13.353961 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.353945    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 21:28:13.354230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.354216    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 21:28:13.358453 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.358430    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:28:13.374817 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.374797    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.374901 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.374829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r89f\" (UniqueName: \"kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.475501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.475474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.475595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.475504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r89f\" (UniqueName: \"kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.478048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.478022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.483007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.482983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r89f\" (UniqueName: \"kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f\") pod \"authorino-7fdccf7bfc-ljbq4\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") " pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.659762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.659691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:28:13.926895 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:13.926826    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:28:13.929698 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:28:13.929674    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f7b2be3_18ed_4bce_ae84_7ffc6627d434.slice/crio-25b597561e9310f89c8ae020d47939328dc4ca38999c9bc6ba6e5f7b713986d6 WatchSource:0}: Error finding container 25b597561e9310f89c8ae020d47939328dc4ca38999c9bc6ba6e5f7b713986d6: Status 404 returned error can't find the container with id 25b597561e9310f89c8ae020d47939328dc4ca38999c9bc6ba6e5f7b713986d6
Sep 10 21:28:14.856667 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:14.856599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" event={"ID":"7f7b2be3-18ed-4bce-ae84-7ffc6627d434","Type":"ContainerStarted","Data":"25b597561e9310f89c8ae020d47939328dc4ca38999c9bc6ba6e5f7b713986d6"}
Sep 10 21:28:16.866395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:16.866364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" event={"ID":"7f7b2be3-18ed-4bce-ae84-7ffc6627d434","Type":"ContainerStarted","Data":"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"}
Sep 10 21:28:16.881299 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:16.881257    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" podStartSLOduration=1.568813819 podStartE2EDuration="3.881242297s" totalImagesPullingTime="2.312428478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:28:16.880423335 +0000 UTC m=+533.375246874" watchObservedRunningTime="2026-09-10 21:28:16.881242297 +0000 UTC m=+533.376065875"
Sep 10 21:28:18.099532 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:18.099494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:18.199756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:18.199728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:18.302815 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:18.302771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:18.364101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:18.364026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:21.663694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:21.663657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:21.845347 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:21.845314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:27.984963 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:27.984929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:28.048156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:28.048127    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:35.959574 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:35.959545    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:36.060550 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:36.060519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:39.196262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:39.196236    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:39.297286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:39.297259    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:51.953181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:51.953146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:52.051648 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:52.051615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.423062 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:53.423028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.524962 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:53.524929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.808144 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:53.808118    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:53.938181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:53.938149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:54.037119 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:54.037087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:28:54.229399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:28:54.229330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:05.570998 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:05.570949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:05.670512 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:05.670481    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:12.237948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:12.237913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:15.435979 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:15.435946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:34.008117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.008083    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"]
Sep 10 21:29:34.011462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.011445    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.014019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.013835    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 21:29:34.014019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.013909    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 21:29:34.014514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.014495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dxtxk\""
Sep 10 21:29:34.014760 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.014740    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 21:29:34.027947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.027921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"]
Sep 10 21:29:34.080704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.080673    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:34.108618 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108589    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psxrf\" (UniqueName: \"kubernetes.io/projected/f03b47d0-7c75-40b4-87eb-76d10a545043-kube-api-access-psxrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.108763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108636    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.108763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.108763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.108763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b47d0-7c75-40b4-87eb-76d10a545043-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.108763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.108757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210076 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210039    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210076 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210071    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210093    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b47d0-7c75-40b4-87eb-76d10a545043-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210124    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psxrf\" (UniqueName: \"kubernetes.io/projected/f03b47d0-7c75-40b4-87eb-76d10a545043-kube-api-access-psxrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210465    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210561 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210524    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.210561 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.210550    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.212550 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.212528    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f03b47d0-7c75-40b4-87eb-76d10a545043-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.212851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.212834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f03b47d0-7c75-40b4-87eb-76d10a545043-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.217718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.217699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psxrf\" (UniqueName: \"kubernetes.io/projected/f03b47d0-7c75-40b4-87eb-76d10a545043-kube-api-access-psxrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x8w7q\" (UID: \"f03b47d0-7c75-40b4-87eb-76d10a545043\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.325143 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.325105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:34.595872 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:34.595847    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"]
Sep 10 21:29:34.598405 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:29:34.598371    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf03b47d0_7c75_40b4_87eb_76d10a545043.slice/crio-5f4710c12b8fc9fc438fb024440d4a85e7755981f0fd8f14a1855c8d7c5da036 WatchSource:0}: Error finding container 5f4710c12b8fc9fc438fb024440d4a85e7755981f0fd8f14a1855c8d7c5da036: Status 404 returned error can't find the container with id 5f4710c12b8fc9fc438fb024440d4a85e7755981f0fd8f14a1855c8d7c5da036
Sep 10 21:29:35.146341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:35.146305    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q" event={"ID":"f03b47d0-7c75-40b4-87eb-76d10a545043","Type":"ContainerStarted","Data":"5f4710c12b8fc9fc438fb024440d4a85e7755981f0fd8f14a1855c8d7c5da036"}
Sep 10 21:29:36.151418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:36.151384    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q" event={"ID":"f03b47d0-7c75-40b4-87eb-76d10a545043","Type":"ContainerStarted","Data":"0842aff202d0b31c8bfb3be0fd9e109779544e07d90ddb69f22a5b7d9580cf5c"}
Sep 10 21:29:36.166939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:36.166890    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q" podStartSLOduration=2.10070414 podStartE2EDuration="3.166875173s" totalImagesPullingTime="1.066171033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:36.166218543 +0000 UTC m=+612.661042082" watchObservedRunningTime="2026-09-10 21:29:36.166875173 +0000 UTC m=+612.661698710"
Sep 10 21:29:39.021366 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.021330    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:39.316040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.315954    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"]
Sep 10 21:29:39.319150 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.319130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.326576 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.326547    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 10 21:29:39.329273 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.329249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"]
Sep 10 21:29:39.340233 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.340210    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:39.452346 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.452489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452356    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.452489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfs7s\" (UniqueName: \"kubernetes.io/projected/997781d6-f276-432d-9f81-882218a62074-kube-api-access-gfs7s\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.452489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452392    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997781d6-f276-432d-9f81-882218a62074-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.452594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452494    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.452594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.452540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.553851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfs7s\" (UniqueName: \"kubernetes.io/projected/997781d6-f276-432d-9f81-882218a62074-kube-api-access-gfs7s\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997781d6-f276-432d-9f81-882218a62074-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.553977    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554307 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.554280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.554290    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.554437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.554402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.556348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.556329    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/997781d6-f276-432d-9f81-882218a62074-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.556670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.556653    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/997781d6-f276-432d-9f81-882218a62074-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.561460 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.561435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfs7s\" (UniqueName: \"kubernetes.io/projected/997781d6-f276-432d-9f81-882218a62074-kube-api-access-gfs7s\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-959xv\" (UID: \"997781d6-f276-432d-9f81-882218a62074\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.635130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.635080    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:39.904988 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:39.904962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"]
Sep 10 21:29:39.907392 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:29:39.907365    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997781d6_f276_432d_9f81_882218a62074.slice/crio-ceda1ca08e35fb4b214f7295d1fc156c37d32de933fc7c46ed9258b2ec448943 WatchSource:0}: Error finding container ceda1ca08e35fb4b214f7295d1fc156c37d32de933fc7c46ed9258b2ec448943: Status 404 returned error can't find the container with id ceda1ca08e35fb4b214f7295d1fc156c37d32de933fc7c46ed9258b2ec448943
Sep 10 21:29:40.169715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:40.169644    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv" event={"ID":"997781d6-f276-432d-9f81-882218a62074","Type":"ContainerStarted","Data":"2c679537136be14012da29a4803a92356d1be319f79b2cdb275ccd1a503837c8"}
Sep 10 21:29:40.169715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:40.169692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv" event={"ID":"997781d6-f276-432d-9f81-882218a62074","Type":"ContainerStarted","Data":"ceda1ca08e35fb4b214f7295d1fc156c37d32de933fc7c46ed9258b2ec448943"}
Sep 10 21:29:40.188024 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:40.187968    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv" podStartSLOduration=1.012682553 podStartE2EDuration="1.187947119s" totalImagesPullingTime="175.264566ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:40.18697174 +0000 UTC m=+616.681795304" watchObservedRunningTime="2026-09-10 21:29:40.187947119 +0000 UTC m=+616.682770658"
Sep 10 21:29:44.325498 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:44.325468    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:44.326109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:44.325524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:44.337672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:44.337652    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:45.199829 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:45.199793    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x8w7q"
Sep 10 21:29:46.012147 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.012120    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"]
Sep 10 21:29:46.015265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.015242    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.017372 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.017351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 21:29:46.026560 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.026540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"]
Sep 10 21:29:46.044979 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.044957    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:29:46.109263 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109236    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.109403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.109403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109287    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8646\" (UniqueName: \"kubernetes.io/projected/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-kube-api-access-s8646\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.109403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.109737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.109737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.109464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8646\" (UniqueName: \"kubernetes.io/projected/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-kube-api-access-s8646\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.210958 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.210910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.211035 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.211020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.212936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.212916    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.213213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.213194    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.218027 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.218008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8646\" (UniqueName: \"kubernetes.io/projected/20d07898-0a93-4cf1-b6c1-3489bfed0e4c-kube-api-access-s8646\") pod \"e2e-trlp-test-simulated-kserve-598466548d-smsx8\" (UID: \"20d07898-0a93-4cf1-b6c1-3489bfed0e4c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.326033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.325999    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:46.593241 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:46.593213    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"]
Sep 10 21:29:46.595721 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:29:46.595699    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d07898_0a93_4cf1_b6c1_3489bfed0e4c.slice/crio-fd9bd113deed7c98ecbef6faf997f18ea16f0b22e8973ecc03426c748a881e40 WatchSource:0}: Error finding container fd9bd113deed7c98ecbef6faf997f18ea16f0b22e8973ecc03426c748a881e40: Status 404 returned error can't find the container with id fd9bd113deed7c98ecbef6faf997f18ea16f0b22e8973ecc03426c748a881e40
Sep 10 21:29:47.198143 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:47.198108    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8" event={"ID":"20d07898-0a93-4cf1-b6c1-3489bfed0e4c","Type":"ContainerStarted","Data":"9d8ec83c1078dc6236efdb9fd3125e4c0ccdafd9007d0b8016608d0848de3986"}
Sep 10 21:29:47.198499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:47.198158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8" event={"ID":"20d07898-0a93-4cf1-b6c1-3489bfed0e4c","Type":"ContainerStarted","Data":"fd9bd113deed7c98ecbef6faf997f18ea16f0b22e8973ecc03426c748a881e40"}
Sep 10 21:29:47.216113 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:47.216062    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8" podStartSLOduration=2.043620506 podStartE2EDuration="2.216046842s" totalImagesPullingTime="172.426336ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:29:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:29:47.214638842 +0000 UTC m=+623.709462374" watchObservedRunningTime="2026-09-10 21:29:47.216046842 +0000 UTC m=+623.710870409"
Sep 10 21:29:49.635884 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:49.635837    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:49.636271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:49.635909    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:49.648510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:49.648479    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:50.221244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:50.221210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-959xv"
Sep 10 21:29:56.326397 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:56.326360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:56.326801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:56.326430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:56.338501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:56.338480    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:29:57.252098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:29:57.252074    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-smsx8"
Sep 10 21:30:02.514866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:30:02.514832    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:30:23.656773 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:30:23.656725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:30:30.037945 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:30:30.037905    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:38.230330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.230296    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9f4574d5b-8b4rc"]
Sep 10 21:31:38.233407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.233391    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.241183 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.241159    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9f4574d5b-8b4rc"]
Sep 10 21:31:38.313368 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.313339    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/41238f45-b705-4f93-a8bf-dc695dc28fb0-tls-cert\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.313499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.313375    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrpf\" (UniqueName: \"kubernetes.io/projected/41238f45-b705-4f93-a8bf-dc695dc28fb0-kube-api-access-6vrpf\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.414594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.414559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/41238f45-b705-4f93-a8bf-dc695dc28fb0-tls-cert\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.414594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.414593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrpf\" (UniqueName: \"kubernetes.io/projected/41238f45-b705-4f93-a8bf-dc695dc28fb0-kube-api-access-6vrpf\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.417171 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.417149    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/41238f45-b705-4f93-a8bf-dc695dc28fb0-tls-cert\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.423662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.423633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrpf\" (UniqueName: \"kubernetes.io/projected/41238f45-b705-4f93-a8bf-dc695dc28fb0-kube-api-access-6vrpf\") pod \"authorino-9f4574d5b-8b4rc\" (UID: \"41238f45-b705-4f93-a8bf-dc695dc28fb0\") " pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.543728 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.543704    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9f4574d5b-8b4rc"
Sep 10 21:31:38.817670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.817600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9f4574d5b-8b4rc"]
Sep 10 21:31:38.820982 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:31:38.820954    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41238f45_b705_4f93_a8bf_dc695dc28fb0.slice/crio-89496a3a287e748915c7e2644509e34a3208f411b7b84acb9a821f5459b59742 WatchSource:0}: Error finding container 89496a3a287e748915c7e2644509e34a3208f411b7b84acb9a821f5459b59742: Status 404 returned error can't find the container with id 89496a3a287e748915c7e2644509e34a3208f411b7b84acb9a821f5459b59742
Sep 10 21:31:38.822332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:38.822318    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 21:31:39.659517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.659475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9f4574d5b-8b4rc" event={"ID":"41238f45-b705-4f93-a8bf-dc695dc28fb0","Type":"ContainerStarted","Data":"82483e0175c9b2f943c201b497d481bce81224420cfbc0421bba76ab2e66ec98"}
Sep 10 21:31:39.659517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.659522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9f4574d5b-8b4rc" event={"ID":"41238f45-b705-4f93-a8bf-dc695dc28fb0","Type":"ContainerStarted","Data":"89496a3a287e748915c7e2644509e34a3208f411b7b84acb9a821f5459b59742"}
Sep 10 21:31:39.677095 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.677048    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9f4574d5b-8b4rc" podStartSLOduration=1.008855675 podStartE2EDuration="1.677030919s" totalImagesPullingTime="668.175244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:31:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:31:39.675960795 +0000 UTC m=+736.170784333" watchObservedRunningTime="2026-09-10 21:31:39.677030919 +0000 UTC m=+736.171854456"
Sep 10 21:31:39.716944 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.716911    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:31:39.717151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.717125    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" podUID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" containerName="authorino" containerID="cri-o://2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b" gracePeriod=30
Sep 10 21:31:39.962507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:39.962471    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:31:40.027345 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.027316    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r89f\" (UniqueName: \"kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f\") pod \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") "
Sep 10 21:31:40.027506 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.027423    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert\") pod \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\" (UID: \"7f7b2be3-18ed-4bce-ae84-7ffc6627d434\") "
Sep 10 21:31:40.030145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.030105    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f" (OuterVolumeSpecName: "kube-api-access-5r89f") pod "7f7b2be3-18ed-4bce-ae84-7ffc6627d434" (UID: "7f7b2be3-18ed-4bce-ae84-7ffc6627d434"). InnerVolumeSpecName "kube-api-access-5r89f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:31:40.037677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.037651    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7f7b2be3-18ed-4bce-ae84-7ffc6627d434" (UID: "7f7b2be3-18ed-4bce-ae84-7ffc6627d434"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 21:31:40.128476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.128436    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-tls-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:31:40.128476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.128478    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r89f\" (UniqueName: \"kubernetes.io/projected/7f7b2be3-18ed-4bce-ae84-7ffc6627d434-kube-api-access-5r89f\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:31:40.310641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.310536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:40.408562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.408531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:40.666094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.666011    2579 generic.go:356] "Generic (PLEG): container finished" podID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" containerID="2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b" exitCode=0
Sep 10 21:31:40.666094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.666046    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4"
Sep 10 21:31:40.666094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.666076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" event={"ID":"7f7b2be3-18ed-4bce-ae84-7ffc6627d434","Type":"ContainerDied","Data":"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"}
Sep 10 21:31:40.666585 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.666107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdccf7bfc-ljbq4" event={"ID":"7f7b2be3-18ed-4bce-ae84-7ffc6627d434","Type":"ContainerDied","Data":"25b597561e9310f89c8ae020d47939328dc4ca38999c9bc6ba6e5f7b713986d6"}
Sep 10 21:31:40.666585 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.666123    2579 scope.go:117] "RemoveContainer" containerID="2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"
Sep 10 21:31:40.675382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.675363    2579 scope.go:117] "RemoveContainer" containerID="2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"
Sep 10 21:31:40.675690 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:31:40.675669    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b\": container with ID starting with 2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b not found: ID does not exist" containerID="2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"
Sep 10 21:31:40.675747 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.675700    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b"} err="failed to get container status \"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b\": rpc error: code = NotFound desc = could not find container \"2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b\": container with ID starting with 2de93299659f96b7b81824bc49c9b2aee16adf7606737141116bd77882ea3b6b not found: ID does not exist"
Sep 10 21:31:40.703311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.703279    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:31:40.724124 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:40.724088    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fdccf7bfc-ljbq4"]
Sep 10 21:31:41.402719 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:41.402692    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:41.493809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:41.493746    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:41.955427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:41.955386    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" path="/var/lib/kubelet/pods/7f7b2be3-18ed-4bce-ae84-7ffc6627d434/volumes"
Sep 10 21:31:56.795849 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:56.795759    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:56.909347 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:56.909289    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:57.085833 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.085697    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:31:57.086033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.086019    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" containerName="authorino"
Sep 10 21:31:57.086101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.086037    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" containerName="authorino"
Sep 10 21:31:57.086162 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.086101    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f7b2be3-18ed-4bce-ae84-7ffc6627d434" containerName="authorino"
Sep 10 21:31:57.088812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.088768    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.091173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.091153    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3984e6-openshift-default-dockercfg-269cx\""
Sep 10 21:31:57.105422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.105399    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:31:57.110393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.110371    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:57.178904 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.178871    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.178910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.178938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgn9\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179181    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.179288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.179254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.280644 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280613    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.280831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280702    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.280831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280731    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.280831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgn9\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.280831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281009 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281009 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281009 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.280907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281270    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281234    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281315    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281536 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.281536 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.281457    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.283325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.283306    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.283424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.283344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.290220 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.290188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.290427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.290399    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgn9\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9\") pod \"e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") " pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.295220 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.295182    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:57.399572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.399503    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:57.674027 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.674003    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:31:57.675706 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:31:57.675675    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2681206b_2754_4b37_ad9d_0306b9775cc6.slice/crio-c47470e0e82ecf2a06c52a13a2f44cf6717c660af6296905ccad91618d428c3a WatchSource:0}: Error finding container c47470e0e82ecf2a06c52a13a2f44cf6717c660af6296905ccad91618d428c3a: Status 404 returned error can't find the container with id c47470e0e82ecf2a06c52a13a2f44cf6717c660af6296905ccad91618d428c3a
Sep 10 21:31:57.678145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.678115    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:31:57.678250 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.678176    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:31:57.678250 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.678204    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:31:57.739579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.739548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerStarted","Data":"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"}
Sep 10 21:31:57.739687 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.739610    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerStarted","Data":"c47470e0e82ecf2a06c52a13a2f44cf6717c660af6296905ccad91618d428c3a"}
Sep 10 21:31:57.764109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:57.764054    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podStartSLOduration=0.764036435 podStartE2EDuration="764.036435ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:31:57.762919352 +0000 UTC m=+754.257742891" watchObservedRunningTime="2026-09-10 21:31:57.764036435 +0000 UTC m=+754.258859975"
Sep 10 21:31:58.208911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.208875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:58.309753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.309722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:58.400249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.400217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:58.405042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.405017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:58.737879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.737848    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:58.743216 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.743193    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:58.744296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:58.744277    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:31:59.812418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:59.812373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:31:59.914306 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:31:59.914279    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:00.972710 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:00.972669    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:01.076221 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.076183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:01.198802 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.198713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:01.691821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.691787    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:01.783919 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.783887    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:01.821813 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.821705    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:32:01.825058 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.825042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:01.827687 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.827668    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3984e6-dockercfg-x2xw9\""
Sep 10 21:32:01.828046 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.828026    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3984e6\""
Sep 10 21:32:01.837441 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.837418    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:32:01.920344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.920299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:01.920523 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:01.920371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swqx9\" (UniqueName: \"kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.021724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.021684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.022135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.021759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swqx9\" (UniqueName: \"kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.022366 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.022344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.030414 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.030392    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swqx9\" (UniqueName: \"kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9\") pod \"payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") " pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.135125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.135049    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:02.404647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.404573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:32:02.407889 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:02.407860    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaab2691_b56f_4ad2_a1ae_72e6f61a1a96.slice/crio-f216ddfbe36f274b05101082b0e76cb3aa26e3374d45c563ff06bcbf009d3072 WatchSource:0}: Error finding container f216ddfbe36f274b05101082b0e76cb3aa26e3374d45c563ff06bcbf009d3072: Status 404 returned error can't find the container with id f216ddfbe36f274b05101082b0e76cb3aa26e3374d45c563ff06bcbf009d3072
Sep 10 21:32:02.761904 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:02.761827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" event={"ID":"daab2691-b56f-4ad2-a1ae-72e6f61a1a96","Type":"ContainerStarted","Data":"f216ddfbe36f274b05101082b0e76cb3aa26e3374d45c563ff06bcbf009d3072"}
Sep 10 21:32:04.400187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.400152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:04.499469 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.499440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:04.630456 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.630425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:04.773192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.773160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" event={"ID":"daab2691-b56f-4ad2-a1ae-72e6f61a1a96","Type":"ContainerStarted","Data":"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"}
Sep 10 21:32:04.773319 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.773262    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:04.795682 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.795520    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" podStartSLOduration=1.512035618 podStartE2EDuration="3.795499477s" totalImagesPullingTime="2.283463859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:04.791064051 +0000 UTC m=+761.285887601" watchObservedRunningTime="2026-09-10 21:32:04.795499477 +0000 UTC m=+761.290323016"
Sep 10 21:32:04.927591 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.927559    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:04.931061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.931038    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:04.933292 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.933265    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e39a0476-openshift-default-dockercfg-rvjqj\""
Sep 10 21:32:04.947997 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.947934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:04.971719 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:04.971684    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:05.050694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.050906 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.050906 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050729    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051034 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051034 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050931    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051034 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.050950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgmqv\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.051101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.051186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.051258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.051215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.151864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.151827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.151864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.151862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.151886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.151908    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.151934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgmqv\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152041    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152063    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152323    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152540 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152520    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152643    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.152706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.152644    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.153066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.153044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.154500 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.154483    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.155156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.155136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.160061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.160031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgmqv\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.160442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.160422    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token\") pod \"e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") " pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.242231 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.242161    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:05.300910 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.300879    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:05.516823 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.516793    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:05.520455 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:05.520425    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d37850_819d_46cf_9636_d2a9b9ae088a.slice/crio-d00123962ca57c18b88cf48ebe6955db8152f8a44eac0821b5b2520682774d16 WatchSource:0}: Error finding container d00123962ca57c18b88cf48ebe6955db8152f8a44eac0821b5b2520682774d16: Status 404 returned error can't find the container with id d00123962ca57c18b88cf48ebe6955db8152f8a44eac0821b5b2520682774d16
Sep 10 21:32:05.522665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.522630    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:05.522756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.522701    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:05.522756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.522740    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:05.563911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.563889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:05.663846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.663811    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:05.780746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.780649    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerStarted","Data":"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"}
Sep 10 21:32:05.780746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.780713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerStarted","Data":"d00123962ca57c18b88cf48ebe6955db8152f8a44eac0821b5b2520682774d16"}
Sep 10 21:32:05.803794 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.803727    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podStartSLOduration=1.80371167 podStartE2EDuration="1.80371167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:05.801427364 +0000 UTC m=+762.296250906" watchObservedRunningTime="2026-09-10 21:32:05.80371167 +0000 UTC m=+762.298535210"
Sep 10 21:32:05.870901 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:05.870870    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:06.242801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:06.242680    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:06.247644 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:06.247616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:06.785227 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:06.785192    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:06.786036 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:06.786015    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:07.169873 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:07.169793    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:07.268448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:07.268416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:08.649684 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:08.649651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:08.745440 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:08.745407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:09.154801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:09.153640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:09.244980 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:09.244919    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:11.204348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:11.204253    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:11.303245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:11.303215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.422597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:12.422558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.526926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:12.526861    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.762888 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:12.762810    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:12.910737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:12.910701    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:13.015763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:13.015736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:15.782442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:15.782409    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:32:15.894219 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:15.894181    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:16.043097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:16.042999    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:16.141349 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:16.141299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:16.320124 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:16.320032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:17.835158 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.835125    2579 generic.go:356] "Generic (PLEG): container finished" podID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerID="d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5" exitCode=0
Sep 10 21:32:17.835598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.835202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerDied","Data":"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"}
Sep 10 21:32:17.835598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.835564    2579 scope.go:117] "RemoveContainer" containerID="d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"
Sep 10 21:32:17.836615 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.836584    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:17.836713 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.836672    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:17.836795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:17.836726    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:18.844704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:18.844656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerStarted","Data":"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"}
Sep 10 21:32:18.939989 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:18.939878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:19.400332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:19.400300    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:32:19.400504 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:19.400404    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 10 21:32:19.400504 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:19.400437    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 10 21:32:20.400313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:20.400275    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 10 21:32:20.400681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:20.400328    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 10 21:32:20.608842 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:20.608803    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:21.399890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:21.399855    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 10 21:32:21.400060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:21.399909    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 10 21:32:22.155386 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:22.155282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:22.254249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:22.254150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:22.385438 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:22.385382    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:22.401199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:22.400868    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 10 21:32:22.401199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:22.400928    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 10 21:32:23.103598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:23.103567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:23.225605 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:23.225558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:23.400152 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:23.400059    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused" start-of-body=
Sep 10 21:32:23.400152 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:23.400123    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.24:15021/healthz/ready\": dial tcp 10.134.0.24:15021: connect: connection refused"
Sep 10 21:32:24.404927 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:24.404883    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:32:24.405426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:24.405404    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:32:24.406007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:24.405988    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:32:24.641835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:24.641765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:24.961996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:24.961961    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:25.226482 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.226409    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:32:25.232209 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.232183    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.234342 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.234309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5bba43-openshift-default-dockercfg-c7jls\""
Sep 10 21:32:25.244159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.244132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:32:25.254158 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.254134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:25.333902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.333863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.333902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.333906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.333926    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.333966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.333992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.334034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smgwf\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.334082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.334116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.334330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.334153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.350065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.350032    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:25.435615 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435672    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435794    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smgwf\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.435932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.436604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.436582    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.437204 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.437176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.438476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.437687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.438476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.438134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.438928 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.438891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.442030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.442006    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.442514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.442477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.444978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.444956    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.445590 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.445567    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smgwf\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf\") pod \"e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") " pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.544313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.544282    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:25.827420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.827389    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:32:25.831574 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:25.831520    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b89f7e6_1e75_4be6_8873_01210381878c.slice/crio-d6d42c61645be175ae413075901f6f2a5a08b0e05811357da6e3cd559dcf4192 WatchSource:0}: Error finding container d6d42c61645be175ae413075901f6f2a5a08b0e05811357da6e3cd559dcf4192: Status 404 returned error can't find the container with id d6d42c61645be175ae413075901f6f2a5a08b0e05811357da6e3cd559dcf4192
Sep 10 21:32:25.834068 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.834031    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:25.834176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.834093    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:25.834176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.834136    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:25.886435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:25.886407    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerStarted","Data":"d6d42c61645be175ae413075901f6f2a5a08b0e05811357da6e3cd559dcf4192"}
Sep 10 21:32:26.892297 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.892221    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerID="21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c" exitCode=0
Sep 10 21:32:26.892818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.892298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerDied","Data":"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"}
Sep 10 21:32:26.892818 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.892725    2579 scope.go:117] "RemoveContainer" containerID="21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"
Sep 10 21:32:26.893894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.893866    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:26.893990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.893952    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:26.894052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.893997    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:26.894104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.893993    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerStarted","Data":"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"}
Sep 10 21:32:26.930392 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:26.930347    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podStartSLOduration=1.930331427 podStartE2EDuration="1.930331427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:26.929470032 +0000 UTC m=+783.424293572" watchObservedRunningTime="2026-09-10 21:32:26.930331427 +0000 UTC m=+783.425154966"
Sep 10 21:32:27.544913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:27.544875    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:27.555315 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:27.555272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:27.900578 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:27.900487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerStarted","Data":"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"}
Sep 10 21:32:27.901043 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:27.900661    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:27.901626 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:27.901611    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:28.242819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.242703    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:28.243012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.242908    2579 patch_prober.go:28] interesting pod/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 10 21:32:28.243012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.242967    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 10 21:32:28.589313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.589271    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:28.706707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.706675    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:28.792938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.792907    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:28.897060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.896984    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:28.992212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:28.992172    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:29.243448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:29.243350    2579 patch_prober.go:28] interesting pod/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 10 21:32:29.243448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:29.243416    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 10 21:32:29.969900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:29.969867    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:30.068685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.068653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:30.141649 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.141607    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:30.243410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.243307    2579 patch_prober.go:28] interesting pod/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 10 21:32:30.243647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.243613    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 10 21:32:30.658685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.658657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:30.756689 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:30.756650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:31.146455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:31.146411    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:31.243156 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:31.243120    2579 patch_prober.go:28] interesting pod/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 10 21:32:31.243413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:31.243385    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 10 21:32:31.839879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:31.839846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:32.243851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.243312    2579 patch_prober.go:28] interesting pod/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused" start-of-body=
Sep 10 21:32:32.243851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.243380    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.26:15021/healthz/ready\": dial tcp 10.134.0.26:15021: connect: connection refused"
Sep 10 21:32:32.684884 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.684848    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"]
Sep 10 21:32:32.688286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.688259    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:32.690363 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.690346    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:32:32.690447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.690351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:32:32.695552 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.695523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"]
Sep 10 21:32:32.741264 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.741101    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:32.804743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.804701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp\" (UID: \"d9faa8a4-8c6f-4823-a059-b122e1290d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:32.905934 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.905902    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp\" (UID: \"d9faa8a4-8c6f-4823-a059-b122e1290d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:32.906499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:32.906475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp\" (UID: \"d9faa8a4-8c6f-4823-a059-b122e1290d35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:33.001728 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.001636    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:33.246645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.246616    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:33.247166 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.246947    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:33.247675 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.247657    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:33.271570 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.271547    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"]
Sep 10 21:32:33.273899 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:33.273878    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9faa8a4_8c6f_4823_a059_b122e1290d35.slice/crio-065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb WatchSource:0}: Error finding container 065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb: Status 404 returned error can't find the container with id 065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb
Sep 10 21:32:33.451900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.451868    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:33.930155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:33.930121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp" event={"ID":"d9faa8a4-8c6f-4823-a059-b122e1290d35","Type":"ContainerStarted","Data":"065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb"}
Sep 10 21:32:34.335212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:34.335177    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:35.022690 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:35.022648    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:35.336976 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:35.336944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:35.438297 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:35.438265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:35.941265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:35.941189    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp" event={"ID":"d9faa8a4-8c6f-4823-a059-b122e1290d35","Type":"ContainerStarted","Data":"23d86b0c2a613624e9385f8fb148cd77ca999673743edd0f36e0e9c4cfe70dff"}
Sep 10 21:32:35.956531 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:35.956486    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp" podStartSLOduration=1.918407115 podStartE2EDuration="3.956467299s" totalImagesPullingTime="2.038060184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:35.954762525 +0000 UTC m=+792.449586063" watchObservedRunningTime="2026-09-10 21:32:35.956467299 +0000 UTC m=+792.451290836"
Sep 10 21:32:36.129938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:36.129909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:36.946828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:36.946795    2579 generic.go:356] "Generic (PLEG): container finished" podID="d9faa8a4-8c6f-4823-a059-b122e1290d35" containerID="23d86b0c2a613624e9385f8fb148cd77ca999673743edd0f36e0e9c4cfe70dff" exitCode=0
Sep 10 21:32:36.947200 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:36.946863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp" event={"ID":"d9faa8a4-8c6f-4823-a059-b122e1290d35","Type":"ContainerDied","Data":"23d86b0c2a613624e9385f8fb148cd77ca999673743edd0f36e0e9c4cfe70dff"}
Sep 10 21:32:38.079327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.079304    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:38.151665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.151636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca\") pod \"d9faa8a4-8c6f-4823-a059-b122e1290d35\" (UID: \"d9faa8a4-8c6f-4823-a059-b122e1290d35\") "
Sep 10 21:32:38.152009 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.151987    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d9faa8a4-8c6f-4823-a059-b122e1290d35" (UID: "d9faa8a4-8c6f-4823-a059-b122e1290d35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:32:38.252572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.252503    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9faa8a4-8c6f-4823-a059-b122e1290d35-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:38.956648 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.956618    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"
Sep 10 21:32:38.956907 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.956659    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp" event={"ID":"d9faa8a4-8c6f-4823-a059-b122e1290d35","Type":"ContainerDied","Data":"065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb"}
Sep 10 21:32:38.956907 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:38.956705    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="065b7bcc1948a19f08af002b52f1c17975f2dbf331eadd05a4180acb8aa6bbdb"
Sep 10 21:32:48.668470 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:48.668437    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:48.998952 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:48.998868    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b89f7e6-1e75-4be6-8873-01210381878c" containerID="29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d" exitCode=0
Sep 10 21:32:48.998952 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:48.998901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerDied","Data":"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"}
Sep 10 21:32:48.999296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:48.999282    2579 scope.go:117] "RemoveContainer" containerID="29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"
Sep 10 21:32:49.000403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.000375    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:49.000467 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.000452    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:49.000513 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.000486    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:49.258040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.257960    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:49.334094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.334062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:49.434499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.434467    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:49.448596 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.448566    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"]
Sep 10 21:32:49.455429 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.455410    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e39a0476-lb4xp"]
Sep 10 21:32:49.501973 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.501945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:49.581825 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.581771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:49.582086 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.582058    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy" containerID="cri-o://01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb" gracePeriod=30
Sep 10 21:32:49.608191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.607745    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:49.959879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:49.958823    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9faa8a4-8c6f-4823-a059-b122e1290d35" path="/var/lib/kubelet/pods/d9faa8a4-8c6f-4823-a059-b122e1290d35/volumes"
Sep 10 21:32:50.005487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.005448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerStarted","Data":"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"}
Sep 10 21:32:50.084708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.084671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:50.176510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.176477    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:50.272452 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.272419    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:50.544734 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.544650    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:50.545352 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.545322    2579 patch_prober.go:28] interesting pod/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 21:32:50.545539 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.545514    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 21:32:50.599502 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.599469    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:32:50.599999 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.599982    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9faa8a4-8c6f-4823-a059-b122e1290d35" containerName="revoke-keys"
Sep 10 21:32:50.600099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.600002    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9faa8a4-8c6f-4823-a059-b122e1290d35" containerName="revoke-keys"
Sep 10 21:32:50.600164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.600103    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9faa8a4-8c6f-4823-a059-b122e1290d35" containerName="revoke-keys"
Sep 10 21:32:50.606445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.606419    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.608928 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.608901    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-30df1990-gw-openshift-default-dockercfg-mwl5z\""
Sep 10 21:32:50.616399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.616376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:32:50.634202 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.634179    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:50.746170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.746133    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:50.770052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.770014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.770239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.770052    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.770995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pblmw\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.775826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.771240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.872828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.872828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.872828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872822    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872846    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.872886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pblmw\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873061 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873345    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873565 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873531    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873596    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873811 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873765    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.873856 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.873821    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.875259 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.875232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.875456 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.875437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.883894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.883873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.884039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.884023    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pblmw\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw\") pod \"e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") " pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:50.918082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:50.918046    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:51.061768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.061724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:51.220408 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.220386    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:32:51.222862 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:51.222831    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08998704_0acc_454c_9642_0a3e28ff8f91.slice/crio-3257eb747a77419fc5954060ca789db655421d7b4c035c08f15ba451361f0c62 WatchSource:0}: Error finding container 3257eb747a77419fc5954060ca789db655421d7b4c035c08f15ba451361f0c62: Status 404 returned error can't find the container with id 3257eb747a77419fc5954060ca789db655421d7b4c035c08f15ba451361f0c62
Sep 10 21:32:51.225040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.225006    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:51.225148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.225081    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:51.225148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.225122    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:32:51.545727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.545275    2579 patch_prober.go:28] interesting pod/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 21:32:51.545727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:51.545346    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 21:32:52.019042 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.019007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerStarted","Data":"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"}
Sep 10 21:32:52.019195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.019062    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerStarted","Data":"3257eb747a77419fc5954060ca789db655421d7b4c035c08f15ba451361f0c62"}
Sep 10 21:32:52.042420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.042370    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podStartSLOduration=2.042329502 podStartE2EDuration="2.042329502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:52.041490806 +0000 UTC m=+808.536314345" watchObservedRunningTime="2026-09-10 21:32:52.042329502 +0000 UTC m=+808.537153041"
Sep 10 21:32:52.545501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.545057    2579 patch_prober.go:28] interesting pod/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 21:32:52.545501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.545118    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 21:32:52.919192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.919099    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:52.924165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:52.924143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:53.023557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.023521    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:53.024455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.024433    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:32:53.488600 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.488572    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:53.544715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.544672    2579 patch_prober.go:28] interesting pod/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 21:32:53.544929 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.544740    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 21:32:53.590258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.590217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:53.751858 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.744913    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:53.842997 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:53.842932    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:54.228306 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.228275    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:54.326874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.326847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:32:54.334715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.334689    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:32:54.367407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.367359    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:32:54.368110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.367708    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.368525 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.368164    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:32:54.371053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.371028    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-30df1990\""
Sep 10 21:32:54.372978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.372959    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.375395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.375277    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-30df1990-dockercfg-j2dps\""
Sep 10 21:32:54.380186 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.380162    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:32:54.506978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.506887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjls8\" (UniqueName: \"kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.507157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.507030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.507157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.507099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.507157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.507130    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjqs8\" (UniqueName: \"kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.545170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.545130    2579 patch_prober.go:28] interesting pod/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body=
Sep 10 21:32:54.545324 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.545213    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused"
Sep 10 21:32:54.608328 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.608295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.608607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.608350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.608607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.608369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjqs8\" (UniqueName: \"kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.608607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.608410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjls8\" (UniqueName: \"kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.608987 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.608964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.609254 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.609236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.620026 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.620000    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjls8\" (UniqueName: \"kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8\") pod \"payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.620118 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.620031    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjqs8\" (UniqueName: \"kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8\") pod \"payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") " pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.681955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.681926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:54.688646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.688620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:32:54.908122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.908097    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:54.980228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:54.980205    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:32:54.982008 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:54.981981    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0237a8f8_a0d8_43bd_9208_70d62a950fdd.slice/crio-7feed7e506a234b42d8fea8fd4b45d2a1b29539c754d7dd2f699ae27d9a3c75b WatchSource:0}: Error finding container 7feed7e506a234b42d8fea8fd4b45d2a1b29539c754d7dd2f699ae27d9a3c75b: Status 404 returned error can't find the container with id 7feed7e506a234b42d8fea8fd4b45d2a1b29539c754d7dd2f699ae27d9a3c75b
Sep 10 21:32:55.010233 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.010212    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:32:55.012335 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012315    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012440 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012424    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012500 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012476    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012553 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012502    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012554    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgmqv\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012667 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012667 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012640    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012699    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket\") pod \"b2d37850-819d-46cf-9636-d2a9b9ae088a\" (UID: \"b2d37850-819d-46cf-9636-d2a9b9ae088a\") "
Sep 10 21:32:55.012770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012697    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:32:55.012938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012907    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:55.013023 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.012998    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:55.013129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.013111    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2d37850-819d-46cf-9636-d2a9b9ae088a-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.013191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.013138    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.013191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.013153    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.013191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.013116    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:55.013332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.013273    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:55.014846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.014822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:55.014937 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.014909    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:32:55.015059 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.015041    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:32:55.015195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.015178    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv" (OuterVolumeSpecName: "kube-api-access-zgmqv") pod "b2d37850-819d-46cf-9636-d2a9b9ae088a" (UID: "b2d37850-819d-46cf-9636-d2a9b9ae088a"). InnerVolumeSpecName "kube-api-access-zgmqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:32:55.017949 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:32:55.017920    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417f2a93_5001_4862_b081_0ecb14305be2.slice/crio-bfe54ce6018c594b5ab0741a5e2b1a9fb3439f7f1d144ef960d9065fb81e2fcc WatchSource:0}: Error finding container bfe54ce6018c594b5ab0741a5e2b1a9fb3439f7f1d144ef960d9065fb81e2fcc: Status 404 returned error can't find the container with id bfe54ce6018c594b5ab0741a5e2b1a9fb3439f7f1d144ef960d9065fb81e2fcc
Sep 10 21:32:55.034234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.033873    2579 generic.go:356] "Generic (PLEG): container finished" podID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerID="01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb" exitCode=0
Sep 10 21:32:55.034234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.033919    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"
Sep 10 21:32:55.034234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.033948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerDied","Data":"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"}
Sep 10 21:32:55.034234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.033997    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq" event={"ID":"b2d37850-819d-46cf-9636-d2a9b9ae088a","Type":"ContainerDied","Data":"d00123962ca57c18b88cf48ebe6955db8152f8a44eac0821b5b2520682774d16"}
Sep 10 21:32:55.034234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.034029    2579 scope.go:117] "RemoveContainer" containerID="01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"
Sep 10 21:32:55.042752 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.042690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" event={"ID":"0237a8f8-a0d8-43bd-9208-70d62a950fdd","Type":"ContainerStarted","Data":"7feed7e506a234b42d8fea8fd4b45d2a1b29539c754d7dd2f699ae27d9a3c75b"}
Sep 10 21:32:55.044869 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.044813    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" event={"ID":"417f2a93-5001-4862-b081-0ecb14305be2","Type":"ContainerStarted","Data":"bfe54ce6018c594b5ab0741a5e2b1a9fb3439f7f1d144ef960d9065fb81e2fcc"}
Sep 10 21:32:55.052833 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.052813    2579 scope.go:117] "RemoveContainer" containerID="21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"
Sep 10 21:32:55.064103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.064060    2579 scope.go:117] "RemoveContainer" containerID="01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"
Sep 10 21:32:55.065496 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:32:55.064481    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb\": container with ID starting with 01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb not found: ID does not exist" containerID="01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"
Sep 10 21:32:55.065579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.065520    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb"} err="failed to get container status \"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb\": rpc error: code = NotFound desc = could not find container \"01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb\": container with ID starting with 01e25787219112af87c6c342c831fe09740dac80a7cb7f6ba444e3d29201fafb not found: ID does not exist"
Sep 10 21:32:55.065579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.065552    2579 scope.go:117] "RemoveContainer" containerID="21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"
Sep 10 21:32:55.065941 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:32:55.065920    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c\": container with ID starting with 21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c not found: ID does not exist" containerID="21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"
Sep 10 21:32:55.066054 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.065946    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c"} err="failed to get container status \"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c\": rpc error: code = NotFound desc = could not find container \"21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c\": container with ID starting with 21693b43589a35930a0f195fb70a2839c92ca9e9098323c12212f99f4aceff2c not found: ID does not exist"
Sep 10 21:32:55.067349 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.067328    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:55.072594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.072571    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e39a0476-openshift-default-79f6b95ddb-j9fpq"]
Sep 10 21:32:55.114528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114498    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.114528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114522    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.114528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114531    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.114528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114539    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgmqv\" (UniqueName: \"kubernetes.io/projected/b2d37850-819d-46cf-9636-d2a9b9ae088a-kube-api-access-zgmqv\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.114910 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114548    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2d37850-819d-46cf-9636-d2a9b9ae088a-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.114910 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.114557    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2d37850-819d-46cf-9636-d2a9b9ae088a-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:32:55.548427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.548403    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:55.548694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.548673    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:55.549413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.549392    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:32:55.957017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:55.956940    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" path="/var/lib/kubelet/pods/b2d37850-819d-46cf-9636-d2a9b9ae088a/volumes"
Sep 10 21:32:56.052007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:56.051969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" event={"ID":"0237a8f8-a0d8-43bd-9208-70d62a950fdd","Type":"ContainerStarted","Data":"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"}
Sep 10 21:32:56.052199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:56.052072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:32:56.053827 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:56.053768    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" event={"ID":"417f2a93-5001-4862-b081-0ecb14305be2","Type":"ContainerStarted","Data":"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"}
Sep 10 21:32:56.067708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:56.067647    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" podStartSLOduration=2.067629994 podStartE2EDuration="2.067629994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:56.066751241 +0000 UTC m=+812.561574789" watchObservedRunningTime="2026-09-10 21:32:56.067629994 +0000 UTC m=+812.562453533"
Sep 10 21:32:56.085221 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:56.085174    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" podStartSLOduration=2.085160788 podStartE2EDuration="2.085160788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:32:56.084056961 +0000 UTC m=+812.578880512" watchObservedRunningTime="2026-09-10 21:32:56.085160788 +0000 UTC m=+812.579984478"
Sep 10 21:32:57.059125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:32:57.059094    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:33:07.061391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:07.061359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:33:08.066761 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:08.066722    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:33:08.750249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:08.750204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:15.142661 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.142625    2579 generic.go:356] "Generic (PLEG): container finished" podID="08998704-0acc-454c-9642-0a3e28ff8f91" containerID="64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14" exitCode=0
Sep 10 21:33:15.143167 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.142702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerDied","Data":"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"}
Sep 10 21:33:15.143232 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.143221    2579 scope.go:117] "RemoveContainer" containerID="64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"
Sep 10 21:33:15.146857 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.144729    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:15.146857 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.144857    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:15.146857 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.146520    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:15.420355 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:15.420274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:16.152509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:16.152466    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerStarted","Data":"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"}
Sep 10 21:33:16.918991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:16.918954    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:16.919159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:16.919121    2579 patch_prober.go:28] interesting pod/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 10 21:33:16.919208 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:16.919168    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 10 21:33:17.918381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:17.918344    2579 patch_prober.go:28] interesting pod/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 10 21:33:17.918863 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:17.918402    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 10 21:33:18.425490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425456    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"]
Sep 10 21:33:18.425819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425804    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:18.425868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425820    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:18.425868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425828    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:18.425868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425833    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:18.425962 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.425897    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:18.428746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.428729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:18.430911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.430885    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:33:18.430911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.430889    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:33:18.436565 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.436535    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"]
Sep 10 21:33:18.533594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.533555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30df1990-6md76\" (UID: \"e8daedb2-a0bd-4617-bc7b-1128ef5c015e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:18.634067 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.634025    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30df1990-6md76\" (UID: \"e8daedb2-a0bd-4617-bc7b-1128ef5c015e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:18.634627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.634609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-30df1990-6md76\" (UID: \"e8daedb2-a0bd-4617-bc7b-1128ef5c015e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:18.741446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.741336    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:18.919120 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.919092    2579 patch_prober.go:28] interesting pod/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 10 21:33:18.919494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:18.919146    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 10 21:33:19.024801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:19.024689    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"]
Sep 10 21:33:19.030854 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:33:19.030820    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8daedb2_a0bd_4617_bc7b_1128ef5c015e.slice/crio-2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8 WatchSource:0}: Error finding container 2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8: Status 404 returned error can't find the container with id 2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8
Sep 10 21:33:19.169258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:19.169222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76" event={"ID":"e8daedb2-a0bd-4617-bc7b-1128ef5c015e","Type":"ContainerStarted","Data":"2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8"}
Sep 10 21:33:19.918383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:19.918344    2579 patch_prober.go:28] interesting pod/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 10 21:33:19.918564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:19.918403    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 10 21:33:20.175561 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:20.175473    2579 generic.go:356] "Generic (PLEG): container finished" podID="e8daedb2-a0bd-4617-bc7b-1128ef5c015e" containerID="7c85b5bba579b63d18501b14c60e7300c0f853c1d173765015c9440e56f09b2a" exitCode=0
Sep 10 21:33:20.175924 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:20.175563    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76" event={"ID":"e8daedb2-a0bd-4617-bc7b-1128ef5c015e","Type":"ContainerDied","Data":"7c85b5bba579b63d18501b14c60e7300c0f853c1d173765015c9440e56f09b2a"}
Sep 10 21:33:20.918558 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:20.918523    2579 patch_prober.go:28] interesting pod/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 10 21:33:20.918732 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:20.918577    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 10 21:33:21.320827 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.320804    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:21.381582 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.381546    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca\") pod \"e8daedb2-a0bd-4617-bc7b-1128ef5c015e\" (UID: \"e8daedb2-a0bd-4617-bc7b-1128ef5c015e\") "
Sep 10 21:33:21.381921 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.381899    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e8daedb2-a0bd-4617-bc7b-1128ef5c015e" (UID: "e8daedb2-a0bd-4617-bc7b-1128ef5c015e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:21.482559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.482494    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8daedb2-a0bd-4617-bc7b-1128ef5c015e-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:21.923128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.923095    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:21.923551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.923530    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:21.924127 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:21.924106    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:22.186620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:22.186530    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"
Sep 10 21:33:22.186620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:22.186561    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76" event={"ID":"e8daedb2-a0bd-4617-bc7b-1128ef5c015e","Type":"ContainerDied","Data":"2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8"}
Sep 10 21:33:22.186620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:22.186619    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ba3bf1ca76727ca730eaa79bf4bccaa4bd28406ffb543c14667b6dd5836fbc8"
Sep 10 21:33:24.247526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.247456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.275033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.275004    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:33:24.275235 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.275211    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" podUID="417f2a93-5001-4862-b081-0ecb14305be2" containerName="payload-processing" containerID="cri-o://216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97" gracePeriod=30
Sep 10 21:33:24.282258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.281595    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:33:24.282258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.281839    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" podUID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" containerName="payload-pre-processing" containerID="cri-o://b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4" gracePeriod=30
Sep 10 21:33:24.318425 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.318394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.392087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.392060    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.459860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.459794    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"]
Sep 10 21:33:24.461700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.461674    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-30df1990-6md76"]
Sep 10 21:33:24.464028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.464000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.523416 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.523386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.552963 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.552939    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:33:24.563124 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.563100    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:33:24.587161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.587136    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:33:24.587332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.587314    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy" containerID="cri-o://69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488" gracePeriod=30
Sep 10 21:33:24.606288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.606264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:24.711594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.711565    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjls8\" (UniqueName: \"kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8\") pod \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") "
Sep 10 21:33:24.711594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.711606    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjqs8\" (UniqueName: \"kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8\") pod \"417f2a93-5001-4862-b081-0ecb14305be2\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") "
Sep 10 21:33:24.711878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.711636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume\") pod \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\" (UID: \"0237a8f8-a0d8-43bd-9208-70d62a950fdd\") "
Sep 10 21:33:24.711878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.711668    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume\") pod \"417f2a93-5001-4862-b081-0ecb14305be2\" (UID: \"417f2a93-5001-4862-b081-0ecb14305be2\") "
Sep 10 21:33:24.712097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.712069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0237a8f8-a0d8-43bd-9208-70d62a950fdd" (UID: "0237a8f8-a0d8-43bd-9208-70d62a950fdd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:24.712141 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.712090    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "417f2a93-5001-4862-b081-0ecb14305be2" (UID: "417f2a93-5001-4862-b081-0ecb14305be2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:24.713872 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.713845    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8" (OuterVolumeSpecName: "kube-api-access-bjls8") pod "0237a8f8-a0d8-43bd-9208-70d62a950fdd" (UID: "0237a8f8-a0d8-43bd-9208-70d62a950fdd"). InnerVolumeSpecName "kube-api-access-bjls8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:24.713973 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.713953    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8" (OuterVolumeSpecName: "kube-api-access-tjqs8") pod "417f2a93-5001-4862-b081-0ecb14305be2" (UID: "417f2a93-5001-4862-b081-0ecb14305be2"). InnerVolumeSpecName "kube-api-access-tjqs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:24.813130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.813061    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjls8\" (UniqueName: \"kubernetes.io/projected/0237a8f8-a0d8-43bd-9208-70d62a950fdd-kube-api-access-bjls8\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:24.813130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.813086    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjqs8\" (UniqueName: \"kubernetes.io/projected/417f2a93-5001-4862-b081-0ecb14305be2-kube-api-access-tjqs8\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:24.813130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.813097    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0237a8f8-a0d8-43bd-9208-70d62a950fdd-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:24.813130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:24.813105    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/417f2a93-5001-4862-b081-0ecb14305be2-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:25.030559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.030529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:25.200986 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.200909    2579 generic.go:356] "Generic (PLEG): container finished" podID="417f2a93-5001-4862-b081-0ecb14305be2" containerID="216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97" exitCode=0
Sep 10 21:33:25.200986 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.200944    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"
Sep 10 21:33:25.201190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.200984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" event={"ID":"417f2a93-5001-4862-b081-0ecb14305be2","Type":"ContainerDied","Data":"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"}
Sep 10 21:33:25.201190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.201034    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6" event={"ID":"417f2a93-5001-4862-b081-0ecb14305be2","Type":"ContainerDied","Data":"bfe54ce6018c594b5ab0741a5e2b1a9fb3439f7f1d144ef960d9065fb81e2fcc"}
Sep 10 21:33:25.201190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.201066    2579 scope.go:117] "RemoveContainer" containerID="216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"
Sep 10 21:33:25.202472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.202277    2579 generic.go:356] "Generic (PLEG): container finished" podID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" containerID="b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4" exitCode=0
Sep 10 21:33:25.202472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.202308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" event={"ID":"0237a8f8-a0d8-43bd-9208-70d62a950fdd","Type":"ContainerDied","Data":"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"}
Sep 10 21:33:25.202472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.202319    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"
Sep 10 21:33:25.202472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.202338    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l" event={"ID":"0237a8f8-a0d8-43bd-9208-70d62a950fdd","Type":"ContainerDied","Data":"7feed7e506a234b42d8fea8fd4b45d2a1b29539c754d7dd2f699ae27d9a3c75b"}
Sep 10 21:33:25.209629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.209510    2579 scope.go:117] "RemoveContainer" containerID="216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"
Sep 10 21:33:25.209880 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:33:25.209852    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97\": container with ID starting with 216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97 not found: ID does not exist" containerID="216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"
Sep 10 21:33:25.209966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.209893    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97"} err="failed to get container status \"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97\": rpc error: code = NotFound desc = could not find container \"216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97\": container with ID starting with 216a40f49147bcdc69a8f16894315dcb3f8e9693d73c1ec03704b7cb14903e97 not found: ID does not exist"
Sep 10 21:33:25.209966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.209917    2579 scope.go:117] "RemoveContainer" containerID="b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"
Sep 10 21:33:25.217836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.217821    2579 scope.go:117] "RemoveContainer" containerID="b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"
Sep 10 21:33:25.218090 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:33:25.218073    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4\": container with ID starting with b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4 not found: ID does not exist" containerID="b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"
Sep 10 21:33:25.218151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.218101    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4"} err="failed to get container status \"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4\": rpc error: code = NotFound desc = could not find container \"b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4\": container with ID starting with b21ce9498a633e168c192107a0daf1188003d7950847826741da940090601df4 not found: ID does not exist"
Sep 10 21:33:25.225373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.225352    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:33:25.232395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.232375    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-30df1990-d4b68c746-tv8d6"]
Sep 10 21:33:25.246413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.246394    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:33:25.252640 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.252619    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-30df1990-54d9f45574-54l4l"]
Sep 10 21:33:25.363339 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363305    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:33:25.363623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363611    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="417f2a93-5001-4862-b081-0ecb14305be2" containerName="payload-processing"
Sep 10 21:33:25.363662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363625    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="417f2a93-5001-4862-b081-0ecb14305be2" containerName="payload-processing"
Sep 10 21:33:25.363662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363635    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" containerName="payload-pre-processing"
Sep 10 21:33:25.363662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363640    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" containerName="payload-pre-processing"
Sep 10 21:33:25.363662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363656    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8daedb2-a0bd-4617-bc7b-1128ef5c015e" containerName="revoke-keys"
Sep 10 21:33:25.363662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363661    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8daedb2-a0bd-4617-bc7b-1128ef5c015e" containerName="revoke-keys"
Sep 10 21:33:25.363832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363708    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" containerName="payload-pre-processing"
Sep 10 21:33:25.363832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363717    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="417f2a93-5001-4862-b081-0ecb14305be2" containerName="payload-processing"
Sep 10 21:33:25.363832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363724    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d37850-819d-46cf-9636-d2a9b9ae088a" containerName="istio-proxy"
Sep 10 21:33:25.363832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.363729    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8daedb2-a0bd-4617-bc7b-1128ef5c015e" containerName="revoke-keys"
Sep 10 21:33:25.368065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.368045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.370528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.370506    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-07c263d6-openshift-default-dockercfg-qsvx5\""
Sep 10 21:33:25.390930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.390894    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:33:25.395742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.395720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:25.505933 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.505842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:25.517602 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517576    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517707    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb25d\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517800    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517822    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.517954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.517861    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618771 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618800    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb25d\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.618926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.618896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619429 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619406    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619483 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.619596 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.619532    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.621515 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.621493    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.621611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.621533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.626375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.626354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb25d\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.626473 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.626423    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token\") pod \"e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") " pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.678293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.678258    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:25.955914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.955888    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0237a8f8-a0d8-43bd-9208-70d62a950fdd" path="/var/lib/kubelet/pods/0237a8f8-a0d8-43bd-9208-70d62a950fdd/volumes"
Sep 10 21:33:25.956209 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:33:25.956187    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2a4a407_7cf4_435f_afe1_bf066e9e9473.slice/crio-d121bdbed1c272f50286a553957bfd5afd0bc6ba18b530e6239cbad164aeeb5e WatchSource:0}: Error finding container d121bdbed1c272f50286a553957bfd5afd0bc6ba18b530e6239cbad164aeeb5e: Status 404 returned error can't find the container with id d121bdbed1c272f50286a553957bfd5afd0bc6ba18b530e6239cbad164aeeb5e
Sep 10 21:33:25.956327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.956312    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="417f2a93-5001-4862-b081-0ecb14305be2" path="/var/lib/kubelet/pods/417f2a93-5001-4862-b081-0ecb14305be2/volumes"
Sep 10 21:33:25.956595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.956584    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8daedb2-a0bd-4617-bc7b-1128ef5c015e" path="/var/lib/kubelet/pods/e8daedb2-a0bd-4617-bc7b-1128ef5c015e/volumes"
Sep 10 21:33:25.956839 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.956825    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:33:25.958232 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.958202    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:25.958290 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.958265    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:25.958329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:25.958293    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:26.208700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:26.208607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerStarted","Data":"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"}
Sep 10 21:33:26.208700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:26.208677    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerStarted","Data":"d121bdbed1c272f50286a553957bfd5afd0bc6ba18b530e6239cbad164aeeb5e"}
Sep 10 21:33:26.230170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:26.230116    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podStartSLOduration=1.230100824 podStartE2EDuration="1.230100824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:26.227577698 +0000 UTC m=+842.722401247" watchObservedRunningTime="2026-09-10 21:33:26.230100824 +0000 UTC m=+842.724924362"
Sep 10 21:33:26.678387 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:26.678352    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:27.628673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:27.628633    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:27.683422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:27.683394    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:27.730320 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:27.730264    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:28.220242 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:28.220208    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:28.221239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:28.221222    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:28.869045 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:28.869014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:28.973799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:28.973753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:29.194942 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.194859    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:29.373163 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.373126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:29.483493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.483421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:29.487598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.487572    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:33:29.491002 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.490982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.494047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.494024    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-07c263d6\""
Sep 10 21:33:29.494147 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.494051    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-07c263d6-dockercfg-hsdmj\""
Sep 10 21:33:29.505386 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.505366    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:33:29.628933 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.628905    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:33:29.633122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.633091    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.652604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.652578    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:33:29.657059 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.657031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.657197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.657068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbcsc\" (UniqueName: \"kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.758496 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.758459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.758637 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.758526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbcsc\" (UniqueName: \"kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.758637 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.758558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdnzk\" (UniqueName: \"kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.758637 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.758599    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.759411 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.759389    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.768647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.768619    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbcsc\" (UniqueName: \"kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc\") pod \"payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.801755 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.801722    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:29.841597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.841568    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:29.859958 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.859928    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdnzk\" (UniqueName: \"kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.860088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.859970    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.860935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.860908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.870450 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.870425    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdnzk\" (UniqueName: \"kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk\") pod \"payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") " pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:29.960559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960531    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960566    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pblmw\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960675    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:33:29.960819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960814    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960874    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.960913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960900    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.961060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960933    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket\") pod \"08998704-0acc-454c-9642-0a3e28ff8f91\" (UID: \"08998704-0acc-454c-9642-0a3e28ff8f91\") "
Sep 10 21:33:29.961060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.960951    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data" (OuterVolumeSpecName: "istio-data") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:33:29.961060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961017    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:33:29.961214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961157    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:29.961268 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961236    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:29.961268 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961256    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:29.961359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961271    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:29.961359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961286    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08998704-0acc-454c-9642-0a3e28ff8f91-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:29.961359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.961291    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:33:29.962943 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.962917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw" (OuterVolumeSpecName: "kube-api-access-pblmw") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "kube-api-access-pblmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:29.963039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.962970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:33:29.963080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.963039    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token" (OuterVolumeSpecName: "istio-token") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:29.963080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.963067    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "08998704-0acc-454c-9642-0a3e28ff8f91" (UID: "08998704-0acc-454c-9642-0a3e28ff8f91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:33:29.978247 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:29.978222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:30.062810 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.062764    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:30.062810 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.062807    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pblmw\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-kube-api-access-pblmw\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:30.063019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.062824    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08998704-0acc-454c-9642-0a3e28ff8f91-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:30.063019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.062838    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08998704-0acc-454c-9642-0a3e28ff8f91-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:30.063019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.062850    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08998704-0acc-454c-9642-0a3e28ff8f91-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:30.077430 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.077409    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:33:30.078885 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:33:30.078857    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a33eaea_0783_4b00_9bd0_82fbd82948a5.slice/crio-cd8a1071cef601dd3a2a28e5e732c40d2fc1d16461cf5bb93986f9f67ea36bea WatchSource:0}: Error finding container cd8a1071cef601dd3a2a28e5e732c40d2fc1d16461cf5bb93986f9f67ea36bea: Status 404 returned error can't find the container with id cd8a1071cef601dd3a2a28e5e732c40d2fc1d16461cf5bb93986f9f67ea36bea
Sep 10 21:33:30.232213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.232183    2579 generic.go:356] "Generic (PLEG): container finished" podID="08998704-0acc-454c-9642-0a3e28ff8f91" containerID="69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488" exitCode=0
Sep 10 21:33:30.232380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.232230    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"
Sep 10 21:33:30.232380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.232258    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerDied","Data":"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"}
Sep 10 21:33:30.232380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.232310    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt" event={"ID":"08998704-0acc-454c-9642-0a3e28ff8f91","Type":"ContainerDied","Data":"3257eb747a77419fc5954060ca789db655421d7b4c035c08f15ba451361f0c62"}
Sep 10 21:33:30.232380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.232340    2579 scope.go:117] "RemoveContainer" containerID="69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"
Sep 10 21:33:30.233982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.233954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" event={"ID":"8a33eaea-0783-4b00-9bd0-82fbd82948a5","Type":"ContainerStarted","Data":"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"}
Sep 10 21:33:30.234194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.234180    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:30.234263 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.234209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" event={"ID":"8a33eaea-0783-4b00-9bd0-82fbd82948a5","Type":"ContainerStarted","Data":"cd8a1071cef601dd3a2a28e5e732c40d2fc1d16461cf5bb93986f9f67ea36bea"}
Sep 10 21:33:30.242194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.242173    2579 scope.go:117] "RemoveContainer" containerID="64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"
Sep 10 21:33:30.251931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.251911    2579 scope.go:117] "RemoveContainer" containerID="69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"
Sep 10 21:33:30.252227 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:33:30.252208    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488\": container with ID starting with 69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488 not found: ID does not exist" containerID="69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"
Sep 10 21:33:30.252275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.252236    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488"} err="failed to get container status \"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488\": rpc error: code = NotFound desc = could not find container \"69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488\": container with ID starting with 69316b4bb24af3f810b4750273547810c9f1884bc3a84de72cdd7a0f6265b488 not found: ID does not exist"
Sep 10 21:33:30.252275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.252251    2579 scope.go:117] "RemoveContainer" containerID="64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"
Sep 10 21:33:30.252525 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:33:30.252508    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14\": container with ID starting with 64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14 not found: ID does not exist" containerID="64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"
Sep 10 21:33:30.252564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.252531    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14"} err="failed to get container status \"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14\": rpc error: code = NotFound desc = could not find container \"64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14\": container with ID starting with 64d7fa79fd3ec7bd3a974c6b9029a57a3e6e39a224ecc4c91d22f616636cde14 not found: ID does not exist"
Sep 10 21:33:30.275279 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.275233    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:33:30.276200 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:33:30.276180    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff510ffe_432f_46f7_b589_331de96d0f59.slice/crio-9fc94151f73ea0c88a158086ef17a0fd8a690c20ab6ee3bf37e77c635fcb2afd WatchSource:0}: Error finding container 9fc94151f73ea0c88a158086ef17a0fd8a690c20ab6ee3bf37e77c635fcb2afd: Status 404 returned error can't find the container with id 9fc94151f73ea0c88a158086ef17a0fd8a690c20ab6ee3bf37e77c635fcb2afd
Sep 10 21:33:30.287495 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.287458    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" podStartSLOduration=1.287444452 podStartE2EDuration="1.287444452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:30.286573881 +0000 UTC m=+846.781397419" watchObservedRunningTime="2026-09-10 21:33:30.287444452 +0000 UTC m=+846.782268002"
Sep 10 21:33:30.308930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.308906    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:33:30.314450 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:30.314426    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-30df1990-gw-openshift-default-f8744ffb-fsbjt"]
Sep 10 21:33:31.242812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:31.242753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" event={"ID":"ff510ffe-432f-46f7-b589-331de96d0f59","Type":"ContainerStarted","Data":"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"}
Sep 10 21:33:31.243307 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:31.242828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" event={"ID":"ff510ffe-432f-46f7-b589-331de96d0f59","Type":"ContainerStarted","Data":"9fc94151f73ea0c88a158086ef17a0fd8a690c20ab6ee3bf37e77c635fcb2afd"}
Sep 10 21:33:31.243307 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:31.243104    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:31.259338 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:31.259288    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" podStartSLOduration=2.259269502 podStartE2EDuration="2.259269502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:31.25796388 +0000 UTC m=+847.752787419" watchObservedRunningTime="2026-09-10 21:33:31.259269502 +0000 UTC m=+847.754093043"
Sep 10 21:33:31.957616 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:31.957585    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" path="/var/lib/kubelet/pods/08998704-0acc-454c-9642-0a3e28ff8f91/volumes"
Sep 10 21:33:41.245994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:41.245959    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:42.250846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:42.249972    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:43.903422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:43.903394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:47.320244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.320161    2579 generic.go:356] "Generic (PLEG): container finished" podID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerID="6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4" exitCode=0
Sep 10 21:33:47.320621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.320238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerDied","Data":"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"}
Sep 10 21:33:47.320621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.320611    2579 scope.go:117] "RemoveContainer" containerID="6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"
Sep 10 21:33:47.321627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.321601    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:47.321677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.321669    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:47.321714 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:47.321699    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:33:48.327082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:48.327045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerStarted","Data":"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"}
Sep 10 21:33:48.679514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:48.679428    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:48.679732 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:48.679638    2579 patch_prober.go:28] interesting pod/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 10 21:33:48.679732 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:48.679682    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 10 21:33:49.678535 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:49.678505    2579 patch_prober.go:28] interesting pod/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 10 21:33:49.678940 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:49.678553    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 10 21:33:50.678719 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:50.678677    2579 patch_prober.go:28] interesting pod/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 10 21:33:50.679129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:50.678731    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 10 21:33:51.678755 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:51.678719    2579 patch_prober.go:28] interesting pod/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 10 21:33:51.679173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:51.678803    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 10 21:33:52.678685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:52.678650    2579 patch_prober.go:28] interesting pod/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 10 21:33:52.678896 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:52.678712    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 10 21:33:53.317669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.315984    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"]
Sep 10 21:33:53.317669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.316480    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:33:53.317669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.316498    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:33:53.317669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.316600    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:33:53.317669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.316612    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:33:53.320598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.319933    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:53.323989 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.323962    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:33:53.324168 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.323962    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:33:53.327406 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.327375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"]
Sep 10 21:33:53.392799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.392408    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw\" (UID: \"6a3a954c-567f-4909-bee0-55770562f2c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:53.493338 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.493298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw\" (UID: \"6a3a954c-567f-4909-bee0-55770562f2c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:53.494109 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.494080    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw\" (UID: \"6a3a954c-567f-4909-bee0-55770562f2c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:53.636327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.636237    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:53.703633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.703196    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:53.704214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.703933    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:53.709018 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.708046    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:33:53.736052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.736028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:53.916924 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:53.916833    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"]
Sep 10 21:33:53.921541 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:33:53.921506    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3a954c_567f_4909_bee0_55770562f2c1.slice/crio-ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001 WatchSource:0}: Error finding container ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001: Status 404 returned error can't find the container with id ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001
Sep 10 21:33:54.373163 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:54.373122    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw" event={"ID":"6a3a954c-567f-4909-bee0-55770562f2c1","Type":"ContainerStarted","Data":"a212d9457dfd69951ce000f65a957fef19b0152c16e1cfc156a8955809a79caf"}
Sep 10 21:33:54.373329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:54.373181    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw" event={"ID":"6a3a954c-567f-4909-bee0-55770562f2c1","Type":"ContainerStarted","Data":"ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001"}
Sep 10 21:33:54.387831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:54.387758    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw" podStartSLOduration=1.38773985 podStartE2EDuration="1.38773985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:33:54.387101259 +0000 UTC m=+870.881924791" watchObservedRunningTime="2026-09-10 21:33:54.38773985 +0000 UTC m=+870.882563389"
Sep 10 21:33:55.205041 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:55.205010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:55.379351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:55.379313    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a3a954c-567f-4909-bee0-55770562f2c1" containerID="a212d9457dfd69951ce000f65a957fef19b0152c16e1cfc156a8955809a79caf" exitCode=0
Sep 10 21:33:55.379519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:55.379403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw" event={"ID":"6a3a954c-567f-4909-bee0-55770562f2c1","Type":"ContainerDied","Data":"a212d9457dfd69951ce000f65a957fef19b0152c16e1cfc156a8955809a79caf"}
Sep 10 21:33:56.509245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:56.509224    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:56.624421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:56.624383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca\") pod \"6a3a954c-567f-4909-bee0-55770562f2c1\" (UID: \"6a3a954c-567f-4909-bee0-55770562f2c1\") "
Sep 10 21:33:56.624791 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:56.624757    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a3a954c-567f-4909-bee0-55770562f2c1" (UID: "6a3a954c-567f-4909-bee0-55770562f2c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:56.725535 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:56.725463    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a3a954c-567f-4909-bee0-55770562f2c1-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:57.390674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:57.390638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw" event={"ID":"6a3a954c-567f-4909-bee0-55770562f2c1","Type":"ContainerDied","Data":"ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001"}
Sep 10 21:33:57.390906 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:57.390700    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab4598e3ed6476be4f67155b62c4950d31e33df890491c41712c5f3a441ab001"
Sep 10 21:33:57.390906 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:57.390659    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"
Sep 10 21:33:59.344507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.344473    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.372081 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.372041    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:33:59.375797 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.372348    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" podUID="ff510ffe-432f-46f7-b589-331de96d0f59" containerName="payload-processing" containerID="cri-o://2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72" gracePeriod=30
Sep 10 21:33:59.385974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.385930    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:33:59.386399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.386331    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" podUID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" containerName="payload-pre-processing" containerID="cri-o://32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e" gracePeriod=30
Sep 10 21:33:59.416636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.416604    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.482572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.482541    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.541894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.541839    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"]
Sep 10 21:33:59.544577 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.544531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.544896 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.544868    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-07c263d6-g2zbw"]
Sep 10 21:33:59.607537 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.607472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:33:59.635840 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.635538    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:33:59.669756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.669732    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:33:59.752823 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.752794    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume\") pod \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") "
Sep 10 21:33:59.752980 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.752863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume\") pod \"ff510ffe-432f-46f7-b589-331de96d0f59\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") "
Sep 10 21:33:59.752980 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.752961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbcsc\" (UniqueName: \"kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc\") pod \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\" (UID: \"8a33eaea-0783-4b00-9bd0-82fbd82948a5\") "
Sep 10 21:33:59.753051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.752987    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdnzk\" (UniqueName: \"kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk\") pod \"ff510ffe-432f-46f7-b589-331de96d0f59\" (UID: \"ff510ffe-432f-46f7-b589-331de96d0f59\") "
Sep 10 21:33:59.753212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.753188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a33eaea-0783-4b00-9bd0-82fbd82948a5" (UID: "8a33eaea-0783-4b00-9bd0-82fbd82948a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:59.753303 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.753266    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ff510ffe-432f-46f7-b589-331de96d0f59" (UID: "ff510ffe-432f-46f7-b589-331de96d0f59"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:33:59.753361 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.753321    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a33eaea-0783-4b00-9bd0-82fbd82948a5-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:59.755200 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.755180    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc" (OuterVolumeSpecName: "kube-api-access-vbcsc") pod "8a33eaea-0783-4b00-9bd0-82fbd82948a5" (UID: "8a33eaea-0783-4b00-9bd0-82fbd82948a5"). InnerVolumeSpecName "kube-api-access-vbcsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:59.755286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.755211    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk" (OuterVolumeSpecName: "kube-api-access-cdnzk") pod "ff510ffe-432f-46f7-b589-331de96d0f59" (UID: "ff510ffe-432f-46f7-b589-331de96d0f59"). InnerVolumeSpecName "kube-api-access-cdnzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:33:59.854148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.854116    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbcsc\" (UniqueName: \"kubernetes.io/projected/8a33eaea-0783-4b00-9bd0-82fbd82948a5-kube-api-access-vbcsc\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:59.854148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.854143    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdnzk\" (UniqueName: \"kubernetes.io/projected/ff510ffe-432f-46f7-b589-331de96d0f59-kube-api-access-cdnzk\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:59.854148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.854154    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff510ffe-432f-46f7-b589-331de96d0f59-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:33:59.955544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:33:59.955468    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a3a954c-567f-4909-bee0-55770562f2c1" path="/var/lib/kubelet/pods/6a3a954c-567f-4909-bee0-55770562f2c1/volumes"
Sep 10 21:34:00.409013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.408978    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" containerID="32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e" exitCode=0
Sep 10 21:34:00.409370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.409018    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"
Sep 10 21:34:00.409370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.409069    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" event={"ID":"8a33eaea-0783-4b00-9bd0-82fbd82948a5","Type":"ContainerDied","Data":"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"}
Sep 10 21:34:00.409370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.409116    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n" event={"ID":"8a33eaea-0783-4b00-9bd0-82fbd82948a5","Type":"ContainerDied","Data":"cd8a1071cef601dd3a2a28e5e732c40d2fc1d16461cf5bb93986f9f67ea36bea"}
Sep 10 21:34:00.409370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.409146    2579 scope.go:117] "RemoveContainer" containerID="32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"
Sep 10 21:34:00.410487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.410460    2579 generic.go:356] "Generic (PLEG): container finished" podID="ff510ffe-432f-46f7-b589-331de96d0f59" containerID="2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72" exitCode=0
Sep 10 21:34:00.410565 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.410492    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" event={"ID":"ff510ffe-432f-46f7-b589-331de96d0f59","Type":"ContainerDied","Data":"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"}
Sep 10 21:34:00.410620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.410559    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr" event={"ID":"ff510ffe-432f-46f7-b589-331de96d0f59","Type":"ContainerDied","Data":"9fc94151f73ea0c88a158086ef17a0fd8a690c20ab6ee3bf37e77c635fcb2afd"}
Sep 10 21:34:00.410620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.410500    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"
Sep 10 21:34:00.417665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.417650    2579 scope.go:117] "RemoveContainer" containerID="32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"
Sep 10 21:34:00.417973 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:00.417953    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e\": container with ID starting with 32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e not found: ID does not exist" containerID="32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"
Sep 10 21:34:00.418037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.417981    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e"} err="failed to get container status \"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e\": rpc error: code = NotFound desc = could not find container \"32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e\": container with ID starting with 32d8a9c41c79b91888d7d9ad125745b3962cef7f70ca6167bd21acc9ce9b862e not found: ID does not exist"
Sep 10 21:34:00.418037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.417997    2579 scope.go:117] "RemoveContainer" containerID="2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"
Sep 10 21:34:00.425485 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.425463    2579 scope.go:117] "RemoveContainer" containerID="2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"
Sep 10 21:34:00.425770 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:00.425753    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72\": container with ID starting with 2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72 not found: ID does not exist" containerID="2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"
Sep 10 21:34:00.425849 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.425791    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72"} err="failed to get container status \"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72\": rpc error: code = NotFound desc = could not find container \"2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72\": container with ID starting with 2d0977db484386e425e24d866a967faed1f964acc40d568449851a871df7de72 not found: ID does not exist"
Sep 10 21:34:00.428572 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.428553    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:34:00.432806 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.432788    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-07c263d6-6765bb85d6-99q5n"]
Sep 10 21:34:00.443238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.443218    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:34:00.445222 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.445201    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-07c263d6-5fb8d887f7-wn4hr"]
Sep 10 21:34:00.632672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.632639    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:34:00.632880 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.632858    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy" containerID="cri-o://82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34" gracePeriod=30
Sep 10 21:34:00.655870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:00.655802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:01.080824 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.080771    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:01.368178 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368102    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:01.368454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368435    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:34:01.368454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368451    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08998704-0acc-454c-9642-0a3e28ff8f91" containerName="istio-proxy"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368464    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff510ffe-432f-46f7-b589-331de96d0f59" containerName="payload-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368469    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff510ffe-432f-46f7-b589-331de96d0f59" containerName="payload-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368479    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a3a954c-567f-4909-bee0-55770562f2c1" containerName="revoke-keys"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368485    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a3a954c-567f-4909-bee0-55770562f2c1" containerName="revoke-keys"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368493    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" containerName="payload-pre-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368498    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" containerName="payload-pre-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368558    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff510ffe-432f-46f7-b589-331de96d0f59" containerName="payload-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368566    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" containerName="payload-pre-processing"
Sep 10 21:34:01.368610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.368572    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a3a954c-567f-4909-bee0-55770562f2c1" containerName="revoke-keys"
Sep 10 21:34:01.372660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.372628    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.374969 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.374953    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-bfdc6d74-openshift-default-dockercfg-g7ddv\""
Sep 10 21:34:01.385269 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.385249    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:01.408129 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.408108    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:01.469716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.469676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.469762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.469827    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qk7\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.469876    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.469901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.470025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.470086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.470139    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.470213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.470176    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.551703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.551679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:01.570885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.570859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.570895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qk7\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.570929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571178    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571291 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571291 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571516 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571501    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571804    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571944 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571834    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.571944 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.571809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.573963 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.573942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.574081 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.573983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.581213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.581180    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qk7\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.581338 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.581245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token\") pod \"e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") " pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.684875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.684765    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:01.957094 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.956686    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a33eaea-0783-4b00-9bd0-82fbd82948a5" path="/var/lib/kubelet/pods/8a33eaea-0783-4b00-9bd0-82fbd82948a5/volumes"
Sep 10 21:34:01.957404 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.957378    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff510ffe-432f-46f7-b589-331de96d0f59" path="/var/lib/kubelet/pods/ff510ffe-432f-46f7-b589-331de96d0f59/volumes"
Sep 10 21:34:01.958929 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.958896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:01.961404 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:01.961379    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bcd3cbd_335c_47f3_8b01_bffc696fde26.slice/crio-82cce795b6b4a12324d04faa6874119fa63127fb30baccb551f017cd530dc427 WatchSource:0}: Error finding container 82cce795b6b4a12324d04faa6874119fa63127fb30baccb551f017cd530dc427: Status 404 returned error can't find the container with id 82cce795b6b4a12324d04faa6874119fa63127fb30baccb551f017cd530dc427
Sep 10 21:34:01.963691 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.963658    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:01.963767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.963722    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:01.963767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:01.963752    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:02.422932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:02.422898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerStarted","Data":"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"}
Sep 10 21:34:02.423117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:02.422969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerStarted","Data":"82cce795b6b4a12324d04faa6874119fa63127fb30baccb551f017cd530dc427"}
Sep 10 21:34:02.443060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:02.443009    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podStartSLOduration=1.44299039 podStartE2EDuration="1.44299039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:02.442212627 +0000 UTC m=+878.937036165" watchObservedRunningTime="2026-09-10 21:34:02.44299039 +0000 UTC m=+878.937813931"
Sep 10 21:34:02.685847 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:02.685725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:03.690459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:03.690430    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:03.795462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:03.795430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:03.892590 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:03.892561    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:04.433065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:04.433029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:04.434165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:04.434146    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:04.971244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:04.971217    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.075041 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.075002    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.366760 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.366726    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.476065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.476017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.549877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.549846    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:05.553254 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.553231    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.553467 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.553447    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:05.555826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.555806    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-bfdc6d74\""
Sep 10 21:34:05.563521 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.563496    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:05.580305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.580282    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:05.583325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.583310    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.585613 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.585595    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-bfdc6d74-dockercfg-9xnmn\""
Sep 10 21:34:05.594829 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.594808    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:05.711102 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.711070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.711265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.711152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh66k\" (UniqueName: \"kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.711265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.711177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbggl\" (UniqueName: \"kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.711265 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.711207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.812348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.812318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.812469 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.812398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh66k\" (UniqueName: \"kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.812469 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.812417    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbggl\" (UniqueName: \"kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.812469 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.812446    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.813113 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.813092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.813223 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.813112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.821964 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.821936    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbggl\" (UniqueName: \"kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl\") pod \"payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.822286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.822253    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh66k\" (UniqueName: \"kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k\") pod \"payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") " pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.864468 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.864445    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:05.892286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.892265    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:05.899735 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:05.899713    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:34:06.014888 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.014856    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.014903    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.014946    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015041    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015068    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015125    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015150    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb25d\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015174    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data\") pod \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\" (UID: \"d2a4a407-7cf4-435f-afe1-bf066e9e9473\") "
Sep 10 21:34:06.015282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015183    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:06.015666 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015450    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:06.015666 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015491    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:06.015666 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015521    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.015863 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015750    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:06.015957 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.015940    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:06.017217 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.017191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:06.017333 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.017311    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d" (OuterVolumeSpecName: "kube-api-access-tb25d") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "kube-api-access-tb25d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:06.017739 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.017719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:34:06.017854 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.017743    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2a4a407-7cf4-435f-afe1-bf066e9e9473" (UID: "d2a4a407-7cf4-435f-afe1-bf066e9e9473"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:06.116415 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116382    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116415 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116410    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116415 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116419    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116429    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116451    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116459    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116467    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb25d\" (UniqueName: \"kubernetes.io/projected/d2a4a407-7cf4-435f-afe1-bf066e9e9473-kube-api-access-tb25d\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.116623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.116475    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2a4a407-7cf4-435f-afe1-bf066e9e9473-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:06.146121 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.146067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:06.147869 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:06.147837    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c777100_4f11_4377_b5ca_fdaf243ef6a6.slice/crio-6756b319738c4af3e8ea54d16db77519d33274647c5d00918b397e357db2bd8d WatchSource:0}: Error finding container 6756b319738c4af3e8ea54d16db77519d33274647c5d00918b397e357db2bd8d: Status 404 returned error can't find the container with id 6756b319738c4af3e8ea54d16db77519d33274647c5d00918b397e357db2bd8d
Sep 10 21:34:06.180931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.180910    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:06.183055 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:06.183032    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704a5a13_be09_4f6c_bc42_06287f3a24b9.slice/crio-3b9f59b9caa8040fb1964d147f310ad7545afddc509848c69d588e26d4e24e77 WatchSource:0}: Error finding container 3b9f59b9caa8040fb1964d147f310ad7545afddc509848c69d588e26d4e24e77: Status 404 returned error can't find the container with id 3b9f59b9caa8040fb1964d147f310ad7545afddc509848c69d588e26d4e24e77
Sep 10 21:34:06.443636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.443533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" event={"ID":"1c777100-4f11-4377-b5ca-fdaf243ef6a6","Type":"ContainerStarted","Data":"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"}
Sep 10 21:34:06.443636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.443599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" event={"ID":"1c777100-4f11-4377-b5ca-fdaf243ef6a6","Type":"ContainerStarted","Data":"6756b319738c4af3e8ea54d16db77519d33274647c5d00918b397e357db2bd8d"}
Sep 10 21:34:06.443886 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.443714    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:06.445422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.445398    2579 generic.go:356] "Generic (PLEG): container finished" podID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerID="82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34" exitCode=0
Sep 10 21:34:06.445549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.445448    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"
Sep 10 21:34:06.445549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.445471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerDied","Data":"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"}
Sep 10 21:34:06.445549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.445521    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx" event={"ID":"d2a4a407-7cf4-435f-afe1-bf066e9e9473","Type":"ContainerDied","Data":"d121bdbed1c272f50286a553957bfd5afd0bc6ba18b530e6239cbad164aeeb5e"}
Sep 10 21:34:06.445763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.445553    2579 scope.go:117] "RemoveContainer" containerID="82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"
Sep 10 21:34:06.447419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.447392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" event={"ID":"704a5a13-be09-4f6c-bc42-06287f3a24b9","Type":"ContainerStarted","Data":"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"}
Sep 10 21:34:06.447526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.447442    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" event={"ID":"704a5a13-be09-4f6c-bc42-06287f3a24b9","Type":"ContainerStarted","Data":"3b9f59b9caa8040fb1964d147f310ad7545afddc509848c69d588e26d4e24e77"}
Sep 10 21:34:06.447584 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.447558    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:06.456364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.456347    2579 scope.go:117] "RemoveContainer" containerID="6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"
Sep 10 21:34:06.464878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.464848    2579 scope.go:117] "RemoveContainer" containerID="82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"
Sep 10 21:34:06.465205 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:06.465177    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34\": container with ID starting with 82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34 not found: ID does not exist" containerID="82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"
Sep 10 21:34:06.465336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.465214    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34"} err="failed to get container status \"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34\": rpc error: code = NotFound desc = could not find container \"82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34\": container with ID starting with 82fe5936f85c1e1f0f836628b16b387744349775103294ae6ef7472c64a8ce34 not found: ID does not exist"
Sep 10 21:34:06.465336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.465235    2579 scope.go:117] "RemoveContainer" containerID="6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"
Sep 10 21:34:06.465577 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:06.465557    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4\": container with ID starting with 6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4 not found: ID does not exist" containerID="6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"
Sep 10 21:34:06.465685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.465580    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4"} err="failed to get container status \"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4\": rpc error: code = NotFound desc = could not find container \"6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4\": container with ID starting with 6e7b63eeb18ecf095da3405d313602de3fade1653e2dbea0b0105f02f68db7b4 not found: ID does not exist"
Sep 10 21:34:06.485410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.485367    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" podStartSLOduration=1.4853499829999999 podStartE2EDuration="1.485349983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:06.484275118 +0000 UTC m=+882.979098664" watchObservedRunningTime="2026-09-10 21:34:06.485349983 +0000 UTC m=+882.980173526"
Sep 10 21:34:06.486497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.486468    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" podStartSLOduration=1.486458605 podStartE2EDuration="1.486458605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:06.464718083 +0000 UTC m=+882.959541627" watchObservedRunningTime="2026-09-10 21:34:06.486458605 +0000 UTC m=+882.981282142"
Sep 10 21:34:06.499881 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.499851    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:34:06.505968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:06.505947    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-07c263d6-openshift-default-6b8cff759d-ns6fx"]
Sep 10 21:34:07.956106 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:07.956073    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" path="/var/lib/kubelet/pods/d2a4a407-7cf4-435f-afe1-bf066e9e9473/volumes"
Sep 10 21:34:09.502926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:09.502897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:15.492499 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:15.492421    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:17.457314 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:17.457280    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:17.457937 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:17.457919    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:20.175418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.175386    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.875428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.875391    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:20.965597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.965559    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"]
Sep 10 21:34:20.965982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.965965    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:20.966080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.965985    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:20.966080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.966006    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:20.966080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.966014    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:20.966232 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.966101    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:20.969124 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.969104    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:20.973055 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.973027    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:34:20.973364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.973337    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:34:20.984011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:20.983992    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"]
Sep 10 21:34:21.034427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.034397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq\" (UID: \"4052dc23-7e25-4b55-809b-25ea07277980\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:21.107487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.107459    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:21.134994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.134917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq\" (UID: \"4052dc23-7e25-4b55-809b-25ea07277980\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:21.135854 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.135829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq\" (UID: \"4052dc23-7e25-4b55-809b-25ea07277980\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:21.279711 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.279680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:21.574526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:21.574498    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"]
Sep 10 21:34:21.576477 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:21.576437    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4052dc23_7e25_4b55_809b_25ea07277980.slice/crio-e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22 WatchSource:0}: Error finding container e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22: Status 404 returned error can't find the container with id e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22
Sep 10 21:34:22.526267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:22.526231    2579 generic.go:356] "Generic (PLEG): container finished" podID="4052dc23-7e25-4b55-809b-25ea07277980" containerID="b23053b306ea939cc1180f3ee847a54381276bf0c971a0c6a6e028d1e737156f" exitCode=0
Sep 10 21:34:22.526624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:22.526274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq" event={"ID":"4052dc23-7e25-4b55-809b-25ea07277980","Type":"ContainerDied","Data":"b23053b306ea939cc1180f3ee847a54381276bf0c971a0c6a6e028d1e737156f"}
Sep 10 21:34:22.526624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:22.526375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq" event={"ID":"4052dc23-7e25-4b55-809b-25ea07277980","Type":"ContainerStarted","Data":"e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22"}
Sep 10 21:34:23.660164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:23.660140    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:23.764236 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:23.764207    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca\") pod \"4052dc23-7e25-4b55-809b-25ea07277980\" (UID: \"4052dc23-7e25-4b55-809b-25ea07277980\") "
Sep 10 21:34:23.764544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:23.764523    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4052dc23-7e25-4b55-809b-25ea07277980" (UID: "4052dc23-7e25-4b55-809b-25ea07277980"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:23.864807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:23.864754    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4052dc23-7e25-4b55-809b-25ea07277980-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:24.537281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.537252    2579 generic.go:356] "Generic (PLEG): container finished" podID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerID="50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9" exitCode=0
Sep 10 21:34:24.537501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.537328    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerDied","Data":"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"}
Sep 10 21:34:24.537835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.537811    2579 scope.go:117] "RemoveContainer" containerID="50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"
Sep 10 21:34:24.538807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.538788    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"
Sep 10 21:34:24.538807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.538800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq" event={"ID":"4052dc23-7e25-4b55-809b-25ea07277980","Type":"ContainerDied","Data":"e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22"}
Sep 10 21:34:24.538932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.538842    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4c264602bbee2e9bfb51ae44f5cb6ed790563a34ea9e7aba87169365682cb22"
Sep 10 21:34:24.539059 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.539027    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:24.539136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.539118    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:24.539188 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:24.539165    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:25.546708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.546676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerStarted","Data":"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"}
Sep 10 21:34:25.685959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.685921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:25.686207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.686182    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:25.686301 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.686242    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:25.865546 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.865444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:25.964050 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:25.964008    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:26.685797 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:26.685741    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:26.686199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:26.685848    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:27.346213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:27.346180    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:27.686017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:27.685931    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:27.686017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:27.685986    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:28.654463 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.654421    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"]
Sep 10 21:34:28.655262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.655238    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4052dc23-7e25-4b55-809b-25ea07277980" containerName="revoke-keys"
Sep 10 21:34:28.655432 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.655420    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4052dc23-7e25-4b55-809b-25ea07277980" containerName="revoke-keys"
Sep 10 21:34:28.655621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.655609    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2a4a407-7cf4-435f-afe1-bf066e9e9473" containerName="istio-proxy"
Sep 10 21:34:28.655721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.655711    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4052dc23-7e25-4b55-809b-25ea07277980" containerName="revoke-keys"
Sep 10 21:34:28.659268 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.659247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:28.663397 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.663375    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:34:28.664072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.664052    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:34:28.666753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.666736    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"]
Sep 10 21:34:28.685625 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.685595    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:28.685722 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.685646    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:28.709620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.709593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr\" (UID: \"686a3353-d947-468e-85e5-d07b7e145f9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:28.810692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.810663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr\" (UID: \"686a3353-d947-468e-85e5-d07b7e145f9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:28.811300 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.811280    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr\" (UID: \"686a3353-d947-468e-85e5-d07b7e145f9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:28.971907 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:28.971824    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:29.244673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.244648    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"]
Sep 10 21:34:29.248045 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:29.248014    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686a3353_d947_468e_85e5_d07b7e145f9c.slice/crio-72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9 WatchSource:0}: Error finding container 72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9: Status 404 returned error can't find the container with id 72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9
Sep 10 21:34:29.575924 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.575895    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr" event={"ID":"686a3353-d947-468e-85e5-d07b7e145f9c","Type":"ContainerStarted","Data":"68e32ecf5c6c866322aeec9c0b3b1a9ad5eff63b8392d10cbf8217e2ad5a33e6"}
Sep 10 21:34:29.576038 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.575945    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr" event={"ID":"686a3353-d947-468e-85e5-d07b7e145f9c","Type":"ContainerStarted","Data":"72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9"}
Sep 10 21:34:29.600376 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.600334    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr" podStartSLOduration=1.600318978 podStartE2EDuration="1.600318978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:29.598846984 +0000 UTC m=+906.093670522" watchObservedRunningTime="2026-09-10 21:34:29.600318978 +0000 UTC m=+906.095142507"
Sep 10 21:34:29.685482 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.685451    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:29.685672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:29.685590    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:30.582532 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:30.582490    2579 generic.go:356] "Generic (PLEG): container finished" podID="686a3353-d947-468e-85e5-d07b7e145f9c" containerID="68e32ecf5c6c866322aeec9c0b3b1a9ad5eff63b8392d10cbf8217e2ad5a33e6" exitCode=0
Sep 10 21:34:30.582947 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:30.582528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr" event={"ID":"686a3353-d947-468e-85e5-d07b7e145f9c","Type":"ContainerDied","Data":"68e32ecf5c6c866322aeec9c0b3b1a9ad5eff63b8392d10cbf8217e2ad5a33e6"}
Sep 10 21:34:30.685851 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:30.685819    2579 patch_prober.go:28] interesting pod/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Sep 10 21:34:30.686015 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:30.685886    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Sep 10 21:34:31.417211 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.417175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:31.689594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.689508    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:31.689993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.689816    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:31.690635 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.690617    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:31.718028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.718004    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:31.840846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.840798    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca\") pod \"686a3353-d947-468e-85e5-d07b7e145f9c\" (UID: \"686a3353-d947-468e-85e5-d07b7e145f9c\") "
Sep 10 21:34:31.841092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.841069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "686a3353-d947-468e-85e5-d07b7e145f9c" (UID: "686a3353-d947-468e-85e5-d07b7e145f9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:31.942543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:31.942464    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/686a3353-d947-468e-85e5-d07b7e145f9c-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:32.030764 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.030738    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:32.132995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.132966    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:32.228221 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.228148    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:32.324338 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.324310    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"]
Sep 10 21:34:32.326071 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.326039    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-4b08e7-8g7fq"]
Sep 10 21:34:32.329494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.329474    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:32.428769 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.428730    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:32.593660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.593623    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr" event={"ID":"686a3353-d947-468e-85e5-d07b7e145f9c","Type":"ContainerDied","Data":"72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9"}
Sep 10 21:34:32.593660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.593650    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"
Sep 10 21:34:32.593910 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.593680    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="72e1332f128de1d8d7e6fd4dc694e7adf5545897fd16f0dc505884a580f837f9"
Sep 10 21:34:32.670878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:32.670847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:33.956126 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:33.956096    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4052dc23-7e25-4b55-809b-25ea07277980" path="/var/lib/kubelet/pods/4052dc23-7e25-4b55-809b-25ea07277980/volumes"
Sep 10 21:34:35.152974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.152945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:35.745448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.745420    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:35.784688 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.784659    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:35.784965 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.784940    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" podUID="704a5a13-be09-4f6c-bc42-06287f3a24b9" containerName="payload-processing" containerID="cri-o://d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464" gracePeriod=30
Sep 10 21:34:35.791327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.791298    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:35.791560 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:35.791530    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" podUID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" containerName="payload-pre-processing" containerID="cri-o://81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca" gracePeriod=30
Sep 10 21:34:36.070741 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.070713    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:36.077633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.077613    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:36.115380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.115346    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"]
Sep 10 21:34:36.117975 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.117954    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-bfdc6d74-zk2rr"]
Sep 10 21:34:36.178321 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178292    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume\") pod \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") "
Sep 10 21:34:36.178692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume\") pod \"704a5a13-be09-4f6c-bc42-06287f3a24b9\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") "
Sep 10 21:34:36.178692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178399    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh66k\" (UniqueName: \"kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k\") pod \"704a5a13-be09-4f6c-bc42-06287f3a24b9\" (UID: \"704a5a13-be09-4f6c-bc42-06287f3a24b9\") "
Sep 10 21:34:36.178692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178419    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbggl\" (UniqueName: \"kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl\") pod \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\" (UID: \"1c777100-4f11-4377-b5ca-fdaf243ef6a6\") "
Sep 10 21:34:36.178692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c777100-4f11-4377-b5ca-fdaf243ef6a6" (UID: "1c777100-4f11-4377-b5ca-fdaf243ef6a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:36.178876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.178821    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "704a5a13-be09-4f6c-bc42-06287f3a24b9" (UID: "704a5a13-be09-4f6c-bc42-06287f3a24b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:36.180708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.180687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k" (OuterVolumeSpecName: "kube-api-access-qh66k") pod "704a5a13-be09-4f6c-bc42-06287f3a24b9" (UID: "704a5a13-be09-4f6c-bc42-06287f3a24b9"). InnerVolumeSpecName "kube-api-access-qh66k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:36.180772 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.180741    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl" (OuterVolumeSpecName: "kube-api-access-vbggl") pod "1c777100-4f11-4377-b5ca-fdaf243ef6a6" (UID: "1c777100-4f11-4377-b5ca-fdaf243ef6a6"). InnerVolumeSpecName "kube-api-access-vbggl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:36.240604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.240572    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:36.240843 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.240820    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy" containerID="cri-o://fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd" gracePeriod=30
Sep 10 21:34:36.279445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.279420    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c777100-4f11-4377-b5ca-fdaf243ef6a6-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:36.279445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.279443    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/704a5a13-be09-4f6c-bc42-06287f3a24b9-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:36.279594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.279453    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh66k\" (UniqueName: \"kubernetes.io/projected/704a5a13-be09-4f6c-bc42-06287f3a24b9-kube-api-access-qh66k\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:36.279594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.279462    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbggl\" (UniqueName: \"kubernetes.io/projected/1c777100-4f11-4377-b5ca-fdaf243ef6a6-kube-api-access-vbggl\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:36.612509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.612473    2579 generic.go:356] "Generic (PLEG): container finished" podID="704a5a13-be09-4f6c-bc42-06287f3a24b9" containerID="d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464" exitCode=0
Sep 10 21:34:36.612670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.612515    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"
Sep 10 21:34:36.612670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.612560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" event={"ID":"704a5a13-be09-4f6c-bc42-06287f3a24b9","Type":"ContainerDied","Data":"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"}
Sep 10 21:34:36.612670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.612604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk" event={"ID":"704a5a13-be09-4f6c-bc42-06287f3a24b9","Type":"ContainerDied","Data":"3b9f59b9caa8040fb1964d147f310ad7545afddc509848c69d588e26d4e24e77"}
Sep 10 21:34:36.612670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.612627    2579 scope.go:117] "RemoveContainer" containerID="d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"
Sep 10 21:34:36.614239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.614164    2579 generic.go:356] "Generic (PLEG): container finished" podID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" containerID="81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca" exitCode=0
Sep 10 21:34:36.614239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.614225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" event={"ID":"1c777100-4f11-4377-b5ca-fdaf243ef6a6","Type":"ContainerDied","Data":"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"}
Sep 10 21:34:36.614239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.614259    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"
Sep 10 21:34:36.614239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.614268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9" event={"ID":"1c777100-4f11-4377-b5ca-fdaf243ef6a6","Type":"ContainerDied","Data":"6756b319738c4af3e8ea54d16db77519d33274647c5d00918b397e357db2bd8d"}
Sep 10 21:34:36.621594 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.621572    2579 scope.go:117] "RemoveContainer" containerID="d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"
Sep 10 21:34:36.621916 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:36.621898    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464\": container with ID starting with d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464 not found: ID does not exist" containerID="d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"
Sep 10 21:34:36.621984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.621924    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464"} err="failed to get container status \"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464\": rpc error: code = NotFound desc = could not find container \"d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464\": container with ID starting with d9c43a1b6a415d43ec1132f1d456abed78e8a1ad30d9fc400bc8e3c5c5493464 not found: ID does not exist"
Sep 10 21:34:36.621984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.621941    2579 scope.go:117] "RemoveContainer" containerID="81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"
Sep 10 21:34:36.629557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.629539    2579 scope.go:117] "RemoveContainer" containerID="81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"
Sep 10 21:34:36.629849 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:36.629828    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca\": container with ID starting with 81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca not found: ID does not exist" containerID="81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"
Sep 10 21:34:36.629908 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.629859    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca"} err="failed to get container status \"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca\": rpc error: code = NotFound desc = could not find container \"81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca\": container with ID starting with 81cddc724c6d62781583da56a60a2341728ee80706671efd45f79532bc1cc9ca not found: ID does not exist"
Sep 10 21:34:36.636144 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.636122    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:36.640601 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.640583    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-bfdc6d74-7fd6979d6f-m5dpk"]
Sep 10 21:34:36.650671 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.650652    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:36.652964 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:36.652944    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-bfdc6d74-8668c8cc6b-dhvn9"]
Sep 10 21:34:37.051422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:37.051394    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:37.955261 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:37.955228    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" path="/var/lib/kubelet/pods/1c777100-4f11-4377-b5ca-fdaf243ef6a6/volumes"
Sep 10 21:34:37.955635 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:37.955516    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686a3353-d947-468e-85e5-d07b7e145f9c" path="/var/lib/kubelet/pods/686a3353-d947-468e-85e5-d07b7e145f9c/volumes"
Sep 10 21:34:37.955836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:37.955823    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704a5a13-be09-4f6c-bc42-06287f3a24b9" path="/var/lib/kubelet/pods/704a5a13-be09-4f6c-bc42-06287f3a24b9/volumes"
Sep 10 21:34:38.483484 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:38.483456    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:39.381276 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:39.381245    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:39.486437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:39.486402    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:39.630673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:39.630646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:40.080249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.080215    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:40.241346 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.241318    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:40.357128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357043    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:34:40.357490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357476    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" containerName="payload-pre-processing"
Sep 10 21:34:40.357544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357494    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" containerName="payload-pre-processing"
Sep 10 21:34:40.357544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357540    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704a5a13-be09-4f6c-bc42-06287f3a24b9" containerName="payload-processing"
Sep 10 21:34:40.357612 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357551    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="704a5a13-be09-4f6c-bc42-06287f3a24b9" containerName="payload-processing"
Sep 10 21:34:40.357612 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357581    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686a3353-d947-468e-85e5-d07b7e145f9c" containerName="revoke-keys"
Sep 10 21:34:40.357612 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357589    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="686a3353-d947-468e-85e5-d07b7e145f9c" containerName="revoke-keys"
Sep 10 21:34:40.357707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357657    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="686a3353-d947-468e-85e5-d07b7e145f9c" containerName="revoke-keys"
Sep 10 21:34:40.357707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357673    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c777100-4f11-4377-b5ca-fdaf243ef6a6" containerName="payload-pre-processing"
Sep 10 21:34:40.357707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.357683    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="704a5a13-be09-4f6c-bc42-06287f3a24b9" containerName="payload-processing"
Sep 10 21:34:40.362459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.362437    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.364675 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.364657    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-610c0f-openshift-default-dockercfg-xj2cn\""
Sep 10 21:34:40.374650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.374630    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:34:40.389588 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.389567    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:40.414908 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.414881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.414921    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.414982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.415311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.415273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7r4b\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516273 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516273 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7r4b\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516481 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516481 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516481 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516667    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516707    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.516960 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.517028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.516959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.517373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.517349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.519125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.519096    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.519329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.519302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.530224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.530202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.530332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.530311    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7r4b\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b\") pod \"e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") " pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.569974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.569946    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:40.672807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.672700    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:40.950976 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.950953    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:34:40.952574 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:40.952553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767b074b_ad2e_4481_9839_5a34256a4241.slice/crio-9d49e9ddb7a37c1904f3a03beaf00869be5938c8c330402b6cf0b967a9998858 WatchSource:0}: Error finding container 9d49e9ddb7a37c1904f3a03beaf00869be5938c8c330402b6cf0b967a9998858: Status 404 returned error can't find the container with id 9d49e9ddb7a37c1904f3a03beaf00869be5938c8c330402b6cf0b967a9998858
Sep 10 21:34:40.954885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.954847    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:40.954933 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.954921    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:40.954968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:40.954959    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:34:41.488039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.488005    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:41.624997 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.624971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625009    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625027    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7qk7\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625067    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625215    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625237    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625261    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket\") pod \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\" (UID: \"9bcd3cbd-335c-47f3-8b01-bffc696fde26\") "
Sep 10 21:34:41.625337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625288    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:41.625610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625466    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:41.625610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625517    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:34:41.625720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625656    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.625720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625670    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:41.625720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625676    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.626077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625728    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.626077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.625741    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:41.627507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.627485    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:34:41.627656 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.627618    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:41.627984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.627953    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:34:41.627984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.627958    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7" (OuterVolumeSpecName: "kube-api-access-j7qk7") pod "9bcd3cbd-335c-47f3-8b01-bffc696fde26" (UID: "9bcd3cbd-335c-47f3-8b01-bffc696fde26"). InnerVolumeSpecName "kube-api-access-j7qk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:34:41.641560 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.641528    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerStarted","Data":"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"}
Sep 10 21:34:41.641690 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.641591    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerStarted","Data":"9d49e9ddb7a37c1904f3a03beaf00869be5938c8c330402b6cf0b967a9998858"}
Sep 10 21:34:41.643173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.643150    2579 generic.go:356] "Generic (PLEG): container finished" podID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerID="fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd" exitCode=0
Sep 10 21:34:41.643278 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.643192    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"
Sep 10 21:34:41.643278 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.643229    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerDied","Data":"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"}
Sep 10 21:34:41.643393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.643287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc" event={"ID":"9bcd3cbd-335c-47f3-8b01-bffc696fde26","Type":"ContainerDied","Data":"82cce795b6b4a12324d04faa6874119fa63127fb30baccb551f017cd530dc427"}
Sep 10 21:34:41.643393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.643318    2579 scope.go:117] "RemoveContainer" containerID="fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"
Sep 10 21:34:41.652086 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.652069    2579 scope.go:117] "RemoveContainer" containerID="50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"
Sep 10 21:34:41.659600 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.659585    2579 scope.go:117] "RemoveContainer" containerID="fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"
Sep 10 21:34:41.659879 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:41.659860    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd\": container with ID starting with fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd not found: ID does not exist" containerID="fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"
Sep 10 21:34:41.659937 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.659887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd"} err="failed to get container status \"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd\": rpc error: code = NotFound desc = could not find container \"fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd\": container with ID starting with fdd3923c8d33194fc9b0a94a3e4237796c7f4e332ce0cd13c62ee31f1d8ba3dd not found: ID does not exist"
Sep 10 21:34:41.659937 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.659903    2579 scope.go:117] "RemoveContainer" containerID="50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"
Sep 10 21:34:41.660178 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:34:41.660160    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9\": container with ID starting with 50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9 not found: ID does not exist" containerID="50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"
Sep 10 21:34:41.660229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.660184    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9"} err="failed to get container status \"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9\": rpc error: code = NotFound desc = could not find container \"50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9\": container with ID starting with 50f71629bba36e5ae46df017836dc5f1d2288edc4faffaa0cf28edf55e5de5f9 not found: ID does not exist"
Sep 10 21:34:41.664067 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.664031    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podStartSLOduration=1.664019443 podStartE2EDuration="1.664019443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:41.661684626 +0000 UTC m=+918.156508192" watchObservedRunningTime="2026-09-10 21:34:41.664019443 +0000 UTC m=+918.158842982"
Sep 10 21:34:41.673413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.673391    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:41.679394 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.679373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:41.688046 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.688027    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-bfdc6d74-openshift-default-f6b4c7fb-hh6lc"]
Sep 10 21:34:41.727294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727272    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.727402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727300    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.727402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727313    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bcd3cbd-335c-47f3-8b01-bffc696fde26-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.727402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727325    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7qk7\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-kube-api-access-j7qk7\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.727402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727339    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.727402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.727352    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bcd3cbd-335c-47f3-8b01-bffc696fde26-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:34:41.956483 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:41.956402    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" path="/var/lib/kubelet/pods/9bcd3cbd-335c-47f3-8b01-bffc696fde26/volumes"
Sep 10 21:34:42.677978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:42.677953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:43.653302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:43.653273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:43.654238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:43.654217    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:34:43.757224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:43.757190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:44.016229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:44.016201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:45.144553 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:45.144519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:45.282579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:45.282549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:45.475442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:45.475366    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:45.999488 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:45.999454    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:46.095329 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.095301    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:46.119128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119096    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:34:46.119476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119459    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.119522 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119479    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.119522 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119487    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.119522 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119493    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.119627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119557    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.119627 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.119565    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bcd3cbd-335c-47f3-8b01-bffc696fde26" containerName="istio-proxy"
Sep 10 21:34:46.123509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.123492    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.125992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.125971    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-610c0f\""
Sep 10 21:34:46.139807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.139758    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:34:46.265668 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.265628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.266204 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.265750    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmwt6\" (UniqueName: \"kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.367291 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.367255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmwt6\" (UniqueName: \"kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.367491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.367336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.368154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.368129    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.379291 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.379262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmwt6\" (UniqueName: \"kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6\") pod \"payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.433639 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.433614    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:46.706888 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:46.706868    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:34:46.708454 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:34:46.708427    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169359fc_def0_449d_bfeb_0cdfeae1147c.slice/crio-04ce3a8dd7d1de70041d94013807d36b0550a2cd37f534c313f894475ad29e90 WatchSource:0}: Error finding container 04ce3a8dd7d1de70041d94013807d36b0550a2cd37f534c313f894475ad29e90: Status 404 returned error can't find the container with id 04ce3a8dd7d1de70041d94013807d36b0550a2cd37f534c313f894475ad29e90
Sep 10 21:34:47.207417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.207370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:47.379540 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.379499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:47.591446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.591409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:47.674421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.674390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" event={"ID":"169359fc-def0-449d-bfeb-0cdfeae1147c","Type":"ContainerStarted","Data":"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"}
Sep 10 21:34:47.674614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.674450    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" event={"ID":"169359fc-def0-449d-bfeb-0cdfeae1147c","Type":"ContainerStarted","Data":"04ce3a8dd7d1de70041d94013807d36b0550a2cd37f534c313f894475ad29e90"}
Sep 10 21:34:47.674614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.674497    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:34:47.694188 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:47.694136    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" podStartSLOduration=1.694115718 podStartE2EDuration="1.694115718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:34:47.69325612 +0000 UTC m=+924.188079658" watchObservedRunningTime="2026-09-10 21:34:47.694115718 +0000 UTC m=+924.188939259"
Sep 10 21:34:48.080428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:48.080392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:34:58.681514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:34:58.681482    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:35:07.777122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.777088    2579 generic.go:356] "Generic (PLEG): container finished" podID="767b074b-ad2e-4481-9839-5a34256a4241" containerID="8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb" exitCode=0
Sep 10 21:35:07.777510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.777129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerDied","Data":"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"}
Sep 10 21:35:07.777586 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.777567    2579 scope.go:117] "RemoveContainer" containerID="8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"
Sep 10 21:35:07.778732 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.778701    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:07.778841 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.778822    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:07.778890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:07.778880    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:08.635948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:08.635920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:08.784579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:08.784544    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerStarted","Data":"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"}
Sep 10 21:35:09.673513 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:09.673472    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:09.674645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:09.674619    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:35:09.674805 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:09.674667    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 10 21:35:10.673929 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:10.673892    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:35:10.674335 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:10.673944    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 10 21:35:11.673455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:11.673417    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:35:11.673638 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:11.673489    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 10 21:35:12.673540 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:12.673504    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:35:12.673984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:12.673560    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 10 21:35:13.654003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:13.653953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:13.673723 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:13.673687    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Sep 10 21:35:13.674102 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:13.673736    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Sep 10 21:35:13.928235 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:13.928151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:14.678068 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:14.678029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:14.678962 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:14.678940    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:17.347767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:17.347736    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:19.976623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:19.976590    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:20.079304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:20.079272    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:20.177432 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:20.177404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:23.475162 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.475128    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:23.587624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.587589    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"]
Sep 10 21:35:23.590985 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.590967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:23.593266 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.593241    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:35:23.593387 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.593355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:35:23.598503 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.598388    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"]
Sep 10 21:35:23.702706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.702683    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-610c0f-gklsm\" (UID: \"052298a7-d44e-47d8-977c-71505c0cd8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:23.803298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.803266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-610c0f-gklsm\" (UID: \"052298a7-d44e-47d8-977c-71505c0cd8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:23.805267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.805250    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:35:23.814413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.814396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-610c0f-gklsm\" (UID: \"052298a7-d44e-47d8-977c-71505c0cd8dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:23.904237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.904203    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:35:23.912657 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:23.912639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:24.182765 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:24.182734    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"]
Sep 10 21:35:24.184846 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:35:24.184814    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052298a7_d44e_47d8_977c_71505c0cd8dc.slice/crio-309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1 WatchSource:0}: Error finding container 309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1: Status 404 returned error can't find the container with id 309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1
Sep 10 21:35:24.862702 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:24.862668    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm" event={"ID":"052298a7-d44e-47d8-977c-71505c0cd8dc","Type":"ContainerStarted","Data":"69bd133d24ecd7e8f345a9a61cb367bb5fe878f115177cc96dfc984bc9dd20d4"}
Sep 10 21:35:24.863170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:24.862727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm" event={"ID":"052298a7-d44e-47d8-977c-71505c0cd8dc","Type":"ContainerStarted","Data":"309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1"}
Sep 10 21:35:24.878185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:24.878136    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm" podStartSLOduration=1.878122029 podStartE2EDuration="1.878122029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:24.876402279 +0000 UTC m=+961.371225818" watchObservedRunningTime="2026-09-10 21:35:24.878122029 +0000 UTC m=+961.372945568"
Sep 10 21:35:25.868494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:25.868453    2579 generic.go:356] "Generic (PLEG): container finished" podID="052298a7-d44e-47d8-977c-71505c0cd8dc" containerID="69bd133d24ecd7e8f345a9a61cb367bb5fe878f115177cc96dfc984bc9dd20d4" exitCode=0
Sep 10 21:35:25.868896 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:25.868531    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm" event={"ID":"052298a7-d44e-47d8-977c-71505c0cd8dc","Type":"ContainerDied","Data":"69bd133d24ecd7e8f345a9a61cb367bb5fe878f115177cc96dfc984bc9dd20d4"}
Sep 10 21:35:27.000511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.000485    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:27.031440 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.031418    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca\") pod \"052298a7-d44e-47d8-977c-71505c0cd8dc\" (UID: \"052298a7-d44e-47d8-977c-71505c0cd8dc\") "
Sep 10 21:35:27.031768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.031747    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "052298a7-d44e-47d8-977c-71505c0cd8dc" (UID: "052298a7-d44e-47d8-977c-71505c0cd8dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:27.132204 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.132132    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/052298a7-d44e-47d8-977c-71505c0cd8dc-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:27.878146 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.878116    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"
Sep 10 21:35:27.878308 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.878112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm" event={"ID":"052298a7-d44e-47d8-977c-71505c0cd8dc","Type":"ContainerDied","Data":"309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1"}
Sep 10 21:35:27.878308 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:27.878252    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="309e2a80f9f81da205ba04b9b94ca4a897c180e080044e6202be04a4a5a7dbc1"
Sep 10 21:35:34.048670 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.048635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:34.072295 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.072266    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:35:34.072526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.072500    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" podUID="169359fc-def0-449d-bfeb-0cdfeae1147c" containerName="payload-pre-processing" containerID="cri-o://31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7" gracePeriod=30
Sep 10 21:35:34.148006 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.147973    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:34.224170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.224134    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"]
Sep 10 21:35:34.227132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.227106    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:34.227580 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.227557    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-610c0f-gklsm"]
Sep 10 21:35:34.294276 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.294244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:34.346465 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.346429    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:35:34.396877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.396848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume\") pod \"169359fc-def0-449d-bfeb-0cdfeae1147c\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") "
Sep 10 21:35:34.397004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.396922    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmwt6\" (UniqueName: \"kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6\") pod \"169359fc-def0-449d-bfeb-0cdfeae1147c\" (UID: \"169359fc-def0-449d-bfeb-0cdfeae1147c\") "
Sep 10 21:35:34.397193 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.397170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "169359fc-def0-449d-bfeb-0cdfeae1147c" (UID: "169359fc-def0-449d-bfeb-0cdfeae1147c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:34.399232 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.399196    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6" (OuterVolumeSpecName: "kube-api-access-tmwt6") pod "169359fc-def0-449d-bfeb-0cdfeae1147c" (UID: "169359fc-def0-449d-bfeb-0cdfeae1147c"). InnerVolumeSpecName "kube-api-access-tmwt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:34.498097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.498062    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/169359fc-def0-449d-bfeb-0cdfeae1147c-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:34.498097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.498090    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmwt6\" (UniqueName: \"kubernetes.io/projected/169359fc-def0-449d-bfeb-0cdfeae1147c-kube-api-access-tmwt6\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:34.623514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.623418    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:34.912816 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.912723    2579 generic.go:356] "Generic (PLEG): container finished" podID="169359fc-def0-449d-bfeb-0cdfeae1147c" containerID="31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7" exitCode=0
Sep 10 21:35:34.912816 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.912773    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"
Sep 10 21:35:34.912995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.912801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" event={"ID":"169359fc-def0-449d-bfeb-0cdfeae1147c","Type":"ContainerDied","Data":"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"}
Sep 10 21:35:34.912995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.912938    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk" event={"ID":"169359fc-def0-449d-bfeb-0cdfeae1147c","Type":"ContainerDied","Data":"04ce3a8dd7d1de70041d94013807d36b0550a2cd37f534c313f894475ad29e90"}
Sep 10 21:35:34.912995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.912960    2579 scope.go:117] "RemoveContainer" containerID="31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"
Sep 10 21:35:34.921922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.921902    2579 scope.go:117] "RemoveContainer" containerID="31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"
Sep 10 21:35:34.922175 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:35:34.922159    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7\": container with ID starting with 31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7 not found: ID does not exist" containerID="31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"
Sep 10 21:35:34.922227 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.922183    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7"} err="failed to get container status \"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7\": rpc error: code = NotFound desc = could not find container \"31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7\": container with ID starting with 31f3cb3d2d5bd73f52283c63188686907ea7cc86750353a6673cd35e1dd9b9c7 not found: ID does not exist"
Sep 10 21:35:34.932893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.932867    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:35:34.936559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:34.936540    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-610c0f-588cb864f8-5rxkk"]
Sep 10 21:35:35.955716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:35.955682    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="052298a7-d44e-47d8-977c-71505c0cd8dc" path="/var/lib/kubelet/pods/052298a7-d44e-47d8-977c-71505c0cd8dc/volumes"
Sep 10 21:35:35.956105 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:35.956063    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="169359fc-def0-449d-bfeb-0cdfeae1147c" path="/var/lib/kubelet/pods/169359fc-def0-449d-bfeb-0cdfeae1147c/volumes"
Sep 10 21:35:35.980350 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:35.980324    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:40.664799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:40.664744    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:40.910408 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:40.910380    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:40.961315 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:40.961216    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:35:40.961509 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:40.961459    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" containerID="cri-o://ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3" gracePeriod=30
Sep 10 21:35:41.018501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.018472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:41.380238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.380198    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:41.665163 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665093    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665432    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052298a7-d44e-47d8-977c-71505c0cd8dc" containerName="revoke-keys"
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665444    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="052298a7-d44e-47d8-977c-71505c0cd8dc" containerName="revoke-keys"
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665453    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169359fc-def0-449d-bfeb-0cdfeae1147c" containerName="payload-pre-processing"
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665460    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="169359fc-def0-449d-bfeb-0cdfeae1147c" containerName="payload-pre-processing"
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665516    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="052298a7-d44e-47d8-977c-71505c0cd8dc" containerName="revoke-keys"
Sep 10 21:35:41.665592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.665527    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="169359fc-def0-449d-bfeb-0cdfeae1147c" containerName="payload-pre-processing"
Sep 10 21:35:41.670145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.670124    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.672429 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.672406    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-295803-openshift-default-dockercfg-2t5zc\""
Sep 10 21:35:41.680492 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.680471    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:35:41.694140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.694121    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:41.760487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760678 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760516    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760678 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jz7c\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760714    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.760837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.761085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.760858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.840028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.839998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:41.862230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862203    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862325    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862531    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jz7c\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862869 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862869 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862696    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.862998 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.862976    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.863070 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.863008    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.863120 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.863101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.864752 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.864730    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.865154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.865134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.870986 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.870969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.871368 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.871349    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jz7c\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c\") pod \"e2e-noten-295803-openshift-default-74b5f556f4-24szl\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") " pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:41.983449 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:41.983365    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:42.148131 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.148100    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:42.213525 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.213495    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:42.272746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.272719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:35:42.274942 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:35:42.274891    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda44df6c_a0e7_41b9_b47b_c8fae3fb2c89.slice/crio-db769339b2c8f5a78ac97b5c8d3cc33e093e6f56024d36b84e381036947023c2 WatchSource:0}: Error finding container db769339b2c8f5a78ac97b5c8d3cc33e093e6f56024d36b84e381036947023c2: Status 404 returned error can't find the container with id db769339b2c8f5a78ac97b5c8d3cc33e093e6f56024d36b84e381036947023c2
Sep 10 21:35:42.277037 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.277000    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:42.277159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.277070    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:42.277159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.277100    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:35:42.328088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.328068    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:42.954279 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.954243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" event={"ID":"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89","Type":"ContainerStarted","Data":"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"}
Sep 10 21:35:42.954764 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.954302    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" event={"ID":"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89","Type":"ContainerStarted","Data":"db769339b2c8f5a78ac97b5c8d3cc33e093e6f56024d36b84e381036947023c2"}
Sep 10 21:35:42.975206 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.975158    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" podStartSLOduration=1.975143586 podStartE2EDuration="1.975143586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:35:42.973641244 +0000 UTC m=+979.468464781" watchObservedRunningTime="2026-09-10 21:35:42.975143586 +0000 UTC m=+979.469967122"
Sep 10 21:35:42.983819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:42.983764    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:43.263999 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:43.263972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:43.360998 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:43.360965    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:43.461114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:43.461087    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:43.989304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:43.989278    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:44.680635 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:44.680601    2579 patch_prober.go:28] interesting pod/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:35:44.680831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:44.680659    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:35:44.963303 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:44.963207    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:44.964393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:44.964368    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:35:46.197614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.197582    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:46.305110 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305082    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305122    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305180    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305206    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305233    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305464 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305464 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305326    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305464 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305355    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7r4b\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305464 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305383    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data\") pod \"767b074b-ad2e-4481-9839-5a34256a4241\" (UID: \"767b074b-ad2e-4481-9839-5a34256a4241\") "
Sep 10 21:35:46.305667 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305531    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:46.305746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305621    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:35:46.305746 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305737    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.305913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data" (OuterVolumeSpecName: "istio-data") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:46.305913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:46.305913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.305793    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:46.307508 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.307477    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:35:46.307508 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.307497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:35:46.307676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.307601    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token" (OuterVolumeSpecName: "istio-token") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:46.307727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.307711    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b" (OuterVolumeSpecName: "kube-api-access-w7r4b") pod "767b074b-ad2e-4481-9839-5a34256a4241" (UID: "767b074b-ad2e-4481-9839-5a34256a4241"). InnerVolumeSpecName "kube-api-access-w7r4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:35:46.407049 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407026    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407049 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407047    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407056    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7r4b\" (UniqueName: \"kubernetes.io/projected/767b074b-ad2e-4481-9839-5a34256a4241-kube-api-access-w7r4b\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407066    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407074    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767b074b-ad2e-4481-9839-5a34256a4241-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407081    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767b074b-ad2e-4481-9839-5a34256a4241-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407091    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.407214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.407098    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767b074b-ad2e-4481-9839-5a34256a4241-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:35:46.974623 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.974593    2579 generic.go:356] "Generic (PLEG): container finished" podID="767b074b-ad2e-4481-9839-5a34256a4241" containerID="ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3" exitCode=0
Sep 10 21:35:46.974799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.974634    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"
Sep 10 21:35:46.974799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.974681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerDied","Data":"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"}
Sep 10 21:35:46.974799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.974725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq" event={"ID":"767b074b-ad2e-4481-9839-5a34256a4241","Type":"ContainerDied","Data":"9d49e9ddb7a37c1904f3a03beaf00869be5938c8c330402b6cf0b967a9998858"}
Sep 10 21:35:46.974799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.974747    2579 scope.go:117] "RemoveContainer" containerID="ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"
Sep 10 21:35:46.983765 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.983746    2579 scope.go:117] "RemoveContainer" containerID="8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"
Sep 10 21:35:46.991874 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.991855    2579 scope.go:117] "RemoveContainer" containerID="ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"
Sep 10 21:35:46.992142 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:35:46.992122    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3\": container with ID starting with ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3 not found: ID does not exist" containerID="ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"
Sep 10 21:35:46.992185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.992151    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3"} err="failed to get container status \"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3\": rpc error: code = NotFound desc = could not find container \"ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3\": container with ID starting with ea6699322092f7ec1b57ee0affa8b0f4f2807fe13d14ad6ab9c9ae8b013301f3 not found: ID does not exist"
Sep 10 21:35:46.992185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.992168    2579 scope.go:117] "RemoveContainer" containerID="8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"
Sep 10 21:35:46.992460 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:35:46.992431    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb\": container with ID starting with 8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb not found: ID does not exist" containerID="8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"
Sep 10 21:35:46.992544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.992468    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb"} err="failed to get container status \"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb\": rpc error: code = NotFound desc = could not find container \"8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb\": container with ID starting with 8ff455d862d4a99090a1e3e40fcd70b5e437ba9addd877b101aa7143b09c0fcb not found: ID does not exist"
Sep 10 21:35:46.995218 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.995178    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:35:46.996854 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:46.996833    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-610c0f-openshift-default-677c568cfb-nn5qq"]
Sep 10 21:35:47.954917 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:47.954887    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="767b074b-ad2e-4481-9839-5a34256a4241" path="/var/lib/kubelet/pods/767b074b-ad2e-4481-9839-5a34256a4241/volumes"
Sep 10 21:35:52.821919 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:52.821884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:53.632011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:53.631983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:59.532703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:59.532667    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:59.648555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:59.648525    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:59.748437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:59.748403    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:35:59.846569 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:35:59.846500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:00.532383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:00.532353    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:00.632955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:00.632921    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:00.731562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:00.731534    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:02.309063 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:02.309019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:07.837126 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:07.837094    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:08.226058 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:08.225980    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:08.299663 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:08.299628    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:08.907981 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:08.907947    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:10.176727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:10.176695    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:11.207576 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.207548    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:11.324372 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.324330    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:36:11.324697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.324645    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerName="istio-proxy" containerID="cri-o://d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05" gracePeriod=30
Sep 10 21:36:11.337435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.337409    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:11.571836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.571802    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:11.697350 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:11.697319    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:12.425169 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.425134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:12.528409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528370    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"]
Sep 10 21:36:12.528693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528682    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:12.528751 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528695    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:12.528751 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528705    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:12.528751 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528710    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:12.528877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.528766    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:12.531900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.531884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:12.533884 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.533858    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:36:12.534030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.533967    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:36:12.538189 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.538166    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"]
Sep 10 21:36:12.634956 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.634919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds\" (UID: \"7dbb52aa-c7cd-4163-9558-e41dd7730246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:12.736474 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.736384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds\" (UID: \"7dbb52aa-c7cd-4163-9558-e41dd7730246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:12.737024 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.737005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds\" (UID: \"7dbb52aa-c7cd-4163-9558-e41dd7730246\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:12.738439 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.738407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:12.847033 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:12.846996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:13.114012 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:13.113983    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"]
Sep 10 21:36:13.117287 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:13.117258    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbb52aa_c7cd_4163_9558_e41dd7730246.slice/crio-8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac WatchSource:0}: Error finding container 8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac: Status 404 returned error can't find the container with id 8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac
Sep 10 21:36:14.105274 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:14.105170    2579 generic.go:356] "Generic (PLEG): container finished" podID="7dbb52aa-c7cd-4163-9558-e41dd7730246" containerID="341ca34cc5b74094bc1986192024ef0a6f27c7950b6fbe4b48dec93de949dcc7" exitCode=0
Sep 10 21:36:14.105274 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:14.105255    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds" event={"ID":"7dbb52aa-c7cd-4163-9558-e41dd7730246","Type":"ContainerDied","Data":"341ca34cc5b74094bc1986192024ef0a6f27c7950b6fbe4b48dec93de949dcc7"}
Sep 10 21:36:14.105679 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:14.105308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds" event={"ID":"7dbb52aa-c7cd-4163-9558-e41dd7730246","Type":"ContainerStarted","Data":"8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac"}
Sep 10 21:36:14.965665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:14.965635    2579 patch_prober.go:28] interesting pod/e2e-noten-295803-openshift-default-74b5f556f4-24szl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:36:14.965860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:14.965690    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:36:15.190379 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.190346    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:15.241208 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.241188    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:15.271190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.271169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:15.355530 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.355498    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca\") pod \"7dbb52aa-c7cd-4163-9558-e41dd7730246\" (UID: \"7dbb52aa-c7cd-4163-9558-e41dd7730246\") "
Sep 10 21:36:15.355866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.355845    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7dbb52aa-c7cd-4163-9558-e41dd7730246" (UID: "7dbb52aa-c7cd-4163-9558-e41dd7730246"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:15.403282 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.403252    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:15.456823 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.456795    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dbb52aa-c7cd-4163-9558-e41dd7730246-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:15.721678 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.721646    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:36:15.722015 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.722000    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dbb52aa-c7cd-4163-9558-e41dd7730246" containerName="revoke-keys"
Sep 10 21:36:15.722015 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.722017    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dbb52aa-c7cd-4163-9558-e41dd7730246" containerName="revoke-keys"
Sep 10 21:36:15.722155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.722097    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dbb52aa-c7cd-4163-9558-e41dd7730246" containerName="revoke-keys"
Sep 10 21:36:15.722155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.722106    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="767b074b-ad2e-4481-9839-5a34256a4241" containerName="istio-proxy"
Sep 10 21:36:15.725066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.725049    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.727352 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.727329    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-92c5c624-openshift-default-dockercfg-4hxh9\""
Sep 10 21:36:15.736843 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.736821    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:36:15.743307 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.743284    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:15.860921 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.860881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.860927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.860966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.860997    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.861058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.861098    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.861136    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.861207    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.861331 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.861248    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg7gl\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.930952 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.930924    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:15.961920 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.961894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962091 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.961939    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962199    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962227    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg7gl\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962313    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962520 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962520 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962499    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.962742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.962732    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.964419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.964394    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.965148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.965126    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.969828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.969804    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:15.970181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:15.970160    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg7gl\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl\") pod \"e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") " pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:16.035652 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.035621    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:16.117131 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.117102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds" event={"ID":"7dbb52aa-c7cd-4163-9558-e41dd7730246","Type":"ContainerDied","Data":"8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac"}
Sep 10 21:36:16.117260 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.117153    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8846320b8b8d17a096c584a852614d6c1bf2d54b012d94b15ef93596bb5515ac"
Sep 10 21:36:16.117260 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.117110    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"
Sep 10 21:36:16.313959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.313935    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:36:16.315366 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:16.315339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0d55cf_8771_4465_9be9_29cf7881a641.slice/crio-b0fbb4ce3cc957a2ae43baa65694944357f58b92281adb7501d3f1fa84ab5615 WatchSource:0}: Error finding container b0fbb4ce3cc957a2ae43baa65694944357f58b92281adb7501d3f1fa84ab5615: Status 404 returned error can't find the container with id b0fbb4ce3cc957a2ae43baa65694944357f58b92281adb7501d3f1fa84ab5615
Sep 10 21:36:16.317621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.317587    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:16.317711 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.317647    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:16.317711 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.317675    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:16.577558 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.577529    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:36:16.667749 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.667749 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667751    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667788    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jz7c\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667833    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667892    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667913    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667938    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.667983    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo\") pod \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\" (UID: \"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89\") "
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668030    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668053    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data" (OuterVolumeSpecName: "istio-data") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668149    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668252    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668402    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668414    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668437    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.668493 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668455    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.668939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.668562    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.670184 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.670161    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c" (OuterVolumeSpecName: "kube-api-access-7jz7c") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "kube-api-access-7jz7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:16.670291 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.670158    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:16.670689 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.670662    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:36:16.670820 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.670669    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token" (OuterVolumeSpecName: "istio-token") pod "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" (UID: "da44df6c-a0e7-41b9-b47b-c8fae3fb2c89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:16.769454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.769426    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jz7c\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-kube-api-access-7jz7c\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.769454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.769451    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.769454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.769460    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.769681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.769470    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.769681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.769478    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:16.772449 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:16.772427    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:17.124181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.124146    2579 generic.go:356] "Generic (PLEG): container finished" podID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerID="d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05" exitCode=0
Sep 10 21:36:17.124385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.124200    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"
Sep 10 21:36:17.124385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.124231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" event={"ID":"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89","Type":"ContainerDied","Data":"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"}
Sep 10 21:36:17.124385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.124279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl" event={"ID":"da44df6c-a0e7-41b9-b47b-c8fae3fb2c89","Type":"ContainerDied","Data":"db769339b2c8f5a78ac97b5c8d3cc33e093e6f56024d36b84e381036947023c2"}
Sep 10 21:36:17.124385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.124305    2579 scope.go:117] "RemoveContainer" containerID="d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"
Sep 10 21:36:17.126005 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.125975    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerStarted","Data":"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"}
Sep 10 21:36:17.126134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.126023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerStarted","Data":"b0fbb4ce3cc957a2ae43baa65694944357f58b92281adb7501d3f1fa84ab5615"}
Sep 10 21:36:17.134183 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.134165    2579 scope.go:117] "RemoveContainer" containerID="d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"
Sep 10 21:36:17.134457 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:17.134437    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05\": container with ID starting with d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05 not found: ID does not exist" containerID="d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"
Sep 10 21:36:17.134544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.134462    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05"} err="failed to get container status \"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05\": rpc error: code = NotFound desc = could not find container \"d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05\": container with ID starting with d2182c0d025f9dfca83480cb30d45386196178605f162e944000b79543b0dc05 not found: ID does not exist"
Sep 10 21:36:17.147326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.147279    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podStartSLOduration=2.147262059 podStartE2EDuration="2.147262059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:17.146594149 +0000 UTC m=+1013.641417687" watchObservedRunningTime="2026-09-10 21:36:17.147262059 +0000 UTC m=+1013.642085600"
Sep 10 21:36:17.159803 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.159763    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:36:17.166128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.166106    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-295803-openshift-default-74b5f556f4-24szl"]
Sep 10 21:36:17.955051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:17.955021    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" path="/var/lib/kubelet/pods/da44df6c-a0e7-41b9-b47b-c8fae3fb2c89/volumes"
Sep 10 21:36:18.035894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:18.035861    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:18.041032 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:18.041009    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:18.132034 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:18.132009    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:18.132890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:18.132866    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:19.083203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:19.083173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:19.183844 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:19.183813    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.232953 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.232923    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.293148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.293112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.393904 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.393878    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.854837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.854806    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.914051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.914017    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:36:20.914510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.914490    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerName="istio-proxy"
Sep 10 21:36:20.914564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.914517    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerName="istio-proxy"
Sep 10 21:36:20.914621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.914609    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="da44df6c-a0e7-41b9-b47b-c8fae3fb2c89" containerName="istio-proxy"
Sep 10 21:36:20.917501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.917486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:20.920382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.920359    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-92c5c624-dockercfg-5hwcx\""
Sep 10 21:36:20.920704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.920689    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-92c5c624\""
Sep 10 21:36:20.936357 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.936335    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:36:20.947552 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.947529    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:20.954312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.954291    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:36:20.957391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.957376    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:20.975051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:20.975026    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:36:21.008693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.008662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n7zh\" (UniqueName: \"kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.009026 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.009005    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.110116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.110022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n7zh\" (UniqueName: \"kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.110116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.110074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.110116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.110095    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4bft\" (UniqueName: \"kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.110116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.110116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.110920 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.110903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.118138 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.118114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n7zh\" (UniqueName: \"kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh\") pod \"payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.210909 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.210882    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4bft\" (UniqueName: \"kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.211047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.210915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.211604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.211586    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.218913 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.218879    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4bft\" (UniqueName: \"kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft\") pod \"payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") " pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.227551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.227532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:21.266825 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.266791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:21.505614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.505587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:36:21.507635 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:21.507608    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5409052_7805_4ef5_ac9b_d0afbaf46349.slice/crio-6ff7c41b47466044ac58e1e5b7919dd46bd5de4ebf2c4cbf65162dfe83c2666a WatchSource:0}: Error finding container 6ff7c41b47466044ac58e1e5b7919dd46bd5de4ebf2c4cbf65162dfe83c2666a: Status 404 returned error can't find the container with id 6ff7c41b47466044ac58e1e5b7919dd46bd5de4ebf2c4cbf65162dfe83c2666a
Sep 10 21:36:21.582359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:21.582337    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:36:21.584761 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:21.584734    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f991922_5049_4efd_aa34_4ff1bfb80706.slice/crio-76ea38ba113316812ddb7df7fc7779434b83d043b1b426e276c3652f84592a0d WatchSource:0}: Error finding container 76ea38ba113316812ddb7df7fc7779434b83d043b1b426e276c3652f84592a0d: Status 404 returned error can't find the container with id 76ea38ba113316812ddb7df7fc7779434b83d043b1b426e276c3652f84592a0d
Sep 10 21:36:22.153343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.153308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" event={"ID":"a5409052-7805-4ef5-ac9b-d0afbaf46349","Type":"ContainerStarted","Data":"55e3d1272b15b63e4a806e7d30a3d31cfdcd8cc089ac14c599a3ee7b68eedc74"}
Sep 10 21:36:22.153529 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.153373    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" event={"ID":"a5409052-7805-4ef5-ac9b-d0afbaf46349","Type":"ContainerStarted","Data":"6ff7c41b47466044ac58e1e5b7919dd46bd5de4ebf2c4cbf65162dfe83c2666a"}
Sep 10 21:36:22.153529 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.153417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:22.155131 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.155102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" event={"ID":"5f991922-5049-4efd-aa34-4ff1bfb80706","Type":"ContainerStarted","Data":"b06faef1b65a69be9691d396c136c0da76ee688c2f8fe125e17e916950dbf6db"}
Sep 10 21:36:22.155251 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.155149    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" event={"ID":"5f991922-5049-4efd-aa34-4ff1bfb80706","Type":"ContainerStarted","Data":"76ea38ba113316812ddb7df7fc7779434b83d043b1b426e276c3652f84592a0d"}
Sep 10 21:36:22.155251 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.155187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:22.168385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.168336    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" podStartSLOduration=2.168321834 podStartE2EDuration="2.168321834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:22.168152647 +0000 UTC m=+1018.662976187" watchObservedRunningTime="2026-09-10 21:36:22.168321834 +0000 UTC m=+1018.663145374"
Sep 10 21:36:22.187000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.186955    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" podStartSLOduration=2.186942984 podStartE2EDuration="2.186942984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:22.185357684 +0000 UTC m=+1018.680181222" watchObservedRunningTime="2026-09-10 21:36:22.186942984 +0000 UTC m=+1018.681766522"
Sep 10 21:36:22.587840 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.587809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:22.687292 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.687261    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:22.787948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:22.787918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:23.252369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.252334    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:36:23.252659 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.252625    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" podUID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" containerName="payload-processing" containerID="cri-o://3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d" gracePeriod=30
Sep 10 21:36:23.261479 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.261453    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:23.362605 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.362565    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:23.413253 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.413223    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"]
Sep 10 21:36:23.415761 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.415733    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-3984e6-f9pds"]
Sep 10 21:36:23.429162 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.429142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:23.509006 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.508935    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:36:23.634184 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.634153    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swqx9\" (UniqueName: \"kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9\") pod \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") "
Sep 10 21:36:23.634554 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.634213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume\") pod \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\" (UID: \"daab2691-b56f-4ad2-a1ae-72e6f61a1a96\") "
Sep 10 21:36:23.634596 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.634546    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "daab2691-b56f-4ad2-a1ae-72e6f61a1a96" (UID: "daab2691-b56f-4ad2-a1ae-72e6f61a1a96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:23.636418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.636400    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9" (OuterVolumeSpecName: "kube-api-access-swqx9") pod "daab2691-b56f-4ad2-a1ae-72e6f61a1a96" (UID: "daab2691-b56f-4ad2-a1ae-72e6f61a1a96"). InnerVolumeSpecName "kube-api-access-swqx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:23.734834 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.734806    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:23.734834 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.734830    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swqx9\" (UniqueName: \"kubernetes.io/projected/daab2691-b56f-4ad2-a1ae-72e6f61a1a96-kube-api-access-swqx9\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:23.791769 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.791740    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:23.956844 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:23.956823    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dbb52aa-c7cd-4163-9558-e41dd7730246" path="/var/lib/kubelet/pods/7dbb52aa-c7cd-4163-9558-e41dd7730246/volumes"
Sep 10 21:36:24.171809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.171686    2579 generic.go:356] "Generic (PLEG): container finished" podID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" containerID="3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d" exitCode=0
Sep 10 21:36:24.171809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.171758    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"
Sep 10 21:36:24.172013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.171851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" event={"ID":"daab2691-b56f-4ad2-a1ae-72e6f61a1a96","Type":"ContainerDied","Data":"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"}
Sep 10 21:36:24.172013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.171913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc" event={"ID":"daab2691-b56f-4ad2-a1ae-72e6f61a1a96","Type":"ContainerDied","Data":"f216ddfbe36f274b05101082b0e76cb3aa26e3374d45c563ff06bcbf009d3072"}
Sep 10 21:36:24.172013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.171941    2579 scope.go:117] "RemoveContainer" containerID="3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"
Sep 10 21:36:24.180180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.180154    2579 scope.go:117] "RemoveContainer" containerID="3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"
Sep 10 21:36:24.180423 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:24.180405    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d\": container with ID starting with 3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d not found: ID does not exist" containerID="3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"
Sep 10 21:36:24.180467 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.180433    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d"} err="failed to get container status \"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d\": rpc error: code = NotFound desc = could not find container \"3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d\": container with ID starting with 3b710c0f2afa85a2b4da08eaa25738c1850d723324d367ff89cf91b7bc79b27d not found: ID does not exist"
Sep 10 21:36:24.185843 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.185821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:36:24.188151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:24.188127    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-3984e6-769f46d84c-shrkc"]
Sep 10 21:36:25.956065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:25.956033    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" path="/var/lib/kubelet/pods/daab2691-b56f-4ad2-a1ae-72e6f61a1a96/volumes"
Sep 10 21:36:29.583103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:29.583069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:29.658878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:29.658842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:29.811849 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:29.811813    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:36:29.812130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:29.812095    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy" containerID="cri-o://a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368" gracePeriod=30
Sep 10 21:36:29.851366 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:29.851267    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:30.170207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.170123    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"]
Sep 10 21:36:30.170450 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.170438    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" containerName="payload-processing"
Sep 10 21:36:30.170492 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.170452    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" containerName="payload-processing"
Sep 10 21:36:30.170534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.170524    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="daab2691-b56f-4ad2-a1ae-72e6f61a1a96" containerName="payload-processing"
Sep 10 21:36:30.175151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.175133    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:30.177377 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.177356    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:36:30.177377 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.177367    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:36:30.182956 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.182933    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"]
Sep 10 21:36:30.293097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.293060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7\" (UID: \"87bc452b-eb17-496a-871a-cd66126bf9b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:30.393768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.393721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7\" (UID: \"87bc452b-eb17-496a-871a-cd66126bf9b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:30.394378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.394356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7\" (UID: \"87bc452b-eb17-496a-871a-cd66126bf9b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:30.445737 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.445652    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:30.486487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.486460    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:30.780211 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:30.780174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"]
Sep 10 21:36:30.782456 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:30.782430    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87bc452b_eb17_496a_871a_cd66126bf9b6.slice/crio-d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422 WatchSource:0}: Error finding container d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422: Status 404 returned error can't find the container with id d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422
Sep 10 21:36:31.209113 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:31.209077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7" event={"ID":"87bc452b-eb17-496a-871a-cd66126bf9b6","Type":"ContainerStarted","Data":"91bc71a85b2cbf240f90280e882de4b870dccbea6a482109d766f80b42d0d152"}
Sep 10 21:36:31.209262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:31.209153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7" event={"ID":"87bc452b-eb17-496a-871a-cd66126bf9b6","Type":"ContainerStarted","Data":"d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422"}
Sep 10 21:36:31.222877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:31.222835    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7" podStartSLOduration=1.222821129 podStartE2EDuration="1.222821129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:31.221987378 +0000 UTC m=+1027.716810910" watchObservedRunningTime="2026-09-10 21:36:31.222821129 +0000 UTC m=+1027.717644668"
Sep 10 21:36:32.215503 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:32.215465    2579 generic.go:356] "Generic (PLEG): container finished" podID="87bc452b-eb17-496a-871a-cd66126bf9b6" containerID="91bc71a85b2cbf240f90280e882de4b870dccbea6a482109d766f80b42d0d152" exitCode=0
Sep 10 21:36:32.215877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:32.215519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7" event={"ID":"87bc452b-eb17-496a-871a-cd66126bf9b6","Type":"ContainerDied","Data":"91bc71a85b2cbf240f90280e882de4b870dccbea6a482109d766f80b42d0d152"}
Sep 10 21:36:33.168811 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.168768    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:36:33.168988 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.168854    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:36:33.376519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.376493    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:33.384084 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.384063    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:33.522068 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.522041    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca\") pod \"87bc452b-eb17-496a-871a-cd66126bf9b6\" (UID: \"87bc452b-eb17-496a-871a-cd66126bf9b6\") "
Sep 10 21:36:33.522365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.522343    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "87bc452b-eb17-496a-871a-cd66126bf9b6" (UID: "87bc452b-eb17-496a-871a-cd66126bf9b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:33.623125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:33.623091    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87bc452b-eb17-496a-871a-cd66126bf9b6-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:34.227281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:34.227193    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7" event={"ID":"87bc452b-eb17-496a-871a-cd66126bf9b6","Type":"ContainerDied","Data":"d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422"}
Sep 10 21:36:34.227281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:34.227234    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"
Sep 10 21:36:34.227281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:34.227252    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d82a7adf7d142d5b1c610b2166696ef08bf382ed0a91238af3a72fecf0d36422"
Sep 10 21:36:35.061824 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.061770    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:36:35.135134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135060    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135097    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135134    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135179    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpgn9\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135204    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135356    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135369    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135413    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135437    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135462    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert\") pod \"2681206b-2754-4b37-ad9d-0306b9775cc6\" (UID: \"2681206b-2754-4b37-ad9d-0306b9775cc6\") "
Sep 10 21:36:35.135812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135567    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:35.135812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135731    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data" (OuterVolumeSpecName: "istio-data") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:35.135993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135830    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.135993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135851    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.135993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135863    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.135993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.135887    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:35.137614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.137581    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:35.137763 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.137729    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token" (OuterVolumeSpecName: "istio-token") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:35.138028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.138011    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9" (OuterVolumeSpecName: "kube-api-access-rpgn9") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "kube-api-access-rpgn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:35.138123 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.138110    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2681206b-2754-4b37-ad9d-0306b9775cc6" (UID: "2681206b-2754-4b37-ad9d-0306b9775cc6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:36:35.234661 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.234633    2579 generic.go:356] "Generic (PLEG): container finished" podID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerID="a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368" exitCode=0
Sep 10 21:36:35.234855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.234690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerDied","Data":"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"}
Sep 10 21:36:35.234855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.234701    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"
Sep 10 21:36:35.234855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.234739    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh" event={"ID":"2681206b-2754-4b37-ad9d-0306b9775cc6","Type":"ContainerDied","Data":"c47470e0e82ecf2a06c52a13a2f44cf6717c660af6296905ccad91618d428c3a"}
Sep 10 21:36:35.234855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.234761    2579 scope.go:117] "RemoveContainer" containerID="a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"
Sep 10 21:36:35.236438 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236416    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.236543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236438    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.236543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236454    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2681206b-2754-4b37-ad9d-0306b9775cc6-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.236543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236466    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.236543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236478    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpgn9\" (UniqueName: \"kubernetes.io/projected/2681206b-2754-4b37-ad9d-0306b9775cc6-kube-api-access-rpgn9\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.236543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.236492    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2681206b-2754-4b37-ad9d-0306b9775cc6-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:35.243332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.243314    2579 scope.go:117] "RemoveContainer" containerID="d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"
Sep 10 21:36:35.251075 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.251055    2579 scope.go:117] "RemoveContainer" containerID="a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"
Sep 10 21:36:35.251371 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:35.251351    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368\": container with ID starting with a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368 not found: ID does not exist" containerID="a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"
Sep 10 21:36:35.251412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.251384    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368"} err="failed to get container status \"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368\": rpc error: code = NotFound desc = could not find container \"a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368\": container with ID starting with a0d475814c1a8f312e68cf4e2bfdd42569af21bc829f2a609602b8757635e368 not found: ID does not exist"
Sep 10 21:36:35.251412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.251408    2579 scope.go:117] "RemoveContainer" containerID="d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"
Sep 10 21:36:35.251667 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:35.251650    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5\": container with ID starting with d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5 not found: ID does not exist" containerID="d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"
Sep 10 21:36:35.251715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.251673    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5"} err="failed to get container status \"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5\": rpc error: code = NotFound desc = could not find container \"d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5\": container with ID starting with d4dc9aca66855e0ef3e05c554e74b4df0a50bff4503b752d4177632beb961aa5 not found: ID does not exist"
Sep 10 21:36:35.259692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.259663    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:36:35.262165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.262145    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3984e6-openshift-default-847fcbc5b-xmmlh"]
Sep 10 21:36:35.955914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:35.955879    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" path="/var/lib/kubelet/pods/2681206b-2754-4b37-ad9d-0306b9775cc6/volumes"
Sep 10 21:36:37.246299 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.246266    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerID="411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05" exitCode=0
Sep 10 21:36:37.246724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.246342    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerDied","Data":"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"}
Sep 10 21:36:37.246768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.246758    2579 scope.go:117] "RemoveContainer" containerID="411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"
Sep 10 21:36:37.247789 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.247750    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:37.247859 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.247842    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:37.247896 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:37.247876    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:36:38.257133 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:38.257085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerStarted","Data":"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"}
Sep 10 21:36:39.036605 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:39.036573    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:39.037117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:39.037030    2579 patch_prober.go:28] interesting pod/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 10 21:36:39.037276 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:39.037254    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 10 21:36:40.037191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.037142    2579 patch_prober.go:28] interesting pod/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 10 21:36:40.037694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.037225    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 10 21:36:40.080039 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.080010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:40.182230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.182201    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:40.868922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.868891    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:40.970246 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:40.970212    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.036167 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.036135    2579 patch_prober.go:28] interesting pod/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 10 21:36:41.036422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.036397    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 10 21:36:41.068703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.068669    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"]
Sep 10 21:36:41.069334 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.069243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.070537 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.070518    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5bba43-qlwk7"]
Sep 10 21:36:41.169226 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.169140    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.251800 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.250228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.443867 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.443750    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:41.960002 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:41.959965    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87bc452b-eb17-496a-871a-cd66126bf9b6" path="/var/lib/kubelet/pods/87bc452b-eb17-496a-871a-cd66126bf9b6/volumes"
Sep 10 21:36:42.036831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:42.036768    2579 patch_prober.go:28] interesting pod/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 10 21:36:42.037024 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:42.036876    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 10 21:36:43.036173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:43.036134    2579 patch_prober.go:28] interesting pod/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Sep 10 21:36:43.036759 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:43.036723    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Sep 10 21:36:44.033255 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.033224    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:44.040020 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.039991    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:44.040427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.040274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:44.041104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.041085    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:36:44.679400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.679364    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:44.773429 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:44.773399    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:45.135348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:45.135314    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:47.453332 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:47.453297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:47.554052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:47.554019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:47.671629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:47.671598    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:36:47.671916 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:47.671891    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy" containerID="cri-o://aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca" gracePeriod=30
Sep 10 21:36:47.695391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:47.695370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:48.291060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:48.291026    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:48.626078 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:48.626000    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:49.958575 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:49.958527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:51.101880 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:51.101842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:51.199681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:51.199653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:51.820004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:51.819969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:51.915086 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:51.915056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:52.919280 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:52.919255    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:36:52.946348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:52.946195    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:53.042266 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.042232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:53.087672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087644    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087712    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087815    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087843    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087871    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087909    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087934    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smgwf\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087963    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:53.087982 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.087978    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs\") pod \"5b89f7e6-1e75-4be6-8873-01210381878c\" (UID: \"5b89f7e6-1e75-4be6-8873-01210381878c\") "
Sep 10 21:36:53.088286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088057    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:53.088286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088153    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:53.088286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088255    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:53.088286 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088268    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:36:53.088494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088348    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.088494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088368    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.088494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088384    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.088494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.088397    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.090284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.090258    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf" (OuterVolumeSpecName: "kube-api-access-smgwf") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "kube-api-access-smgwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:53.090631 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.090613    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:36:53.090700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.090640    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:36:53.090700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.090671    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b89f7e6-1e75-4be6-8873-01210381878c" (UID: "5b89f7e6-1e75-4be6-8873-01210381878c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:36:53.189301 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.189267    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.189301 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.189296    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smgwf\" (UniqueName: \"kubernetes.io/projected/5b89f7e6-1e75-4be6-8873-01210381878c-kube-api-access-smgwf\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.189497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.189311    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b89f7e6-1e75-4be6-8873-01210381878c-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.189497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.189324    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b89f7e6-1e75-4be6-8873-01210381878c-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.189497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.189336    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b89f7e6-1e75-4be6-8873-01210381878c-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:36:53.339722 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.339687    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b89f7e6-1e75-4be6-8873-01210381878c" containerID="aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca" exitCode=0
Sep 10 21:36:53.339915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.339730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerDied","Data":"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"}
Sep 10 21:36:53.339915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.339738    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"
Sep 10 21:36:53.339915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.339805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn" event={"ID":"5b89f7e6-1e75-4be6-8873-01210381878c","Type":"ContainerDied","Data":"d6d42c61645be175ae413075901f6f2a5a08b0e05811357da6e3cd559dcf4192"}
Sep 10 21:36:53.339915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.339838    2579 scope.go:117] "RemoveContainer" containerID="aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"
Sep 10 21:36:53.349967 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.349950    2579 scope.go:117] "RemoveContainer" containerID="29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"
Sep 10 21:36:53.359549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.359533    2579 scope.go:117] "RemoveContainer" containerID="aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"
Sep 10 21:36:53.359838 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:53.359815    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca\": container with ID starting with aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca not found: ID does not exist" containerID="aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"
Sep 10 21:36:53.359921 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.359850    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca"} err="failed to get container status \"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca\": rpc error: code = NotFound desc = could not find container \"aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca\": container with ID starting with aaf0055baf481966ec18751af94c7c5ab40faaf540b4722776586ddbd635ceca not found: ID does not exist"
Sep 10 21:36:53.359921 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.359875    2579 scope.go:117] "RemoveContainer" containerID="29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"
Sep 10 21:36:53.360153 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:36:53.360132    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d\": container with ID starting with 29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d not found: ID does not exist" containerID="29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"
Sep 10 21:36:53.360262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.360157    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d"} err="failed to get container status \"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d\": rpc error: code = NotFound desc = could not find container \"29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d\": container with ID starting with 29d84d691aa8517084d69396efc57327ecea68df67b7cc11044095374009e18d not found: ID does not exist"
Sep 10 21:36:53.370794 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.370746    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:36:53.378005 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.377979    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5bba43-openshift-default-689457c5f8-xqdtn"]
Sep 10 21:36:53.410614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.410573    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:53.508820 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.508765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:53.557173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557149    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:36:53.557501 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557489    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557503    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557525    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87bc452b-eb17-496a-871a-cd66126bf9b6" containerName="revoke-keys"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557532    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="87bc452b-eb17-496a-871a-cd66126bf9b6" containerName="revoke-keys"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557538    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557543    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.557551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557549    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557555    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557617    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557627    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557632    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557638    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="87bc452b-eb17-496a-871a-cd66126bf9b6" containerName="revoke-keys"
Sep 10 21:36:53.557762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.557643    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2681206b-2754-4b37-ad9d-0306b9775cc6" containerName="istio-proxy"
Sep 10 21:36:53.561976 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.561962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.563881 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.563861    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-2d9e68\""
Sep 10 21:36:53.563996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.563931    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-2d9e68-dockercfg-mpds9\""
Sep 10 21:36:53.568305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.568286    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:36:53.693629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.693543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.693629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.693600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhbk\" (UniqueName: \"kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.794669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.794632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhbk\" (UniqueName: \"kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.794850 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.794709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.795474 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.795451    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.802418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.802393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhbk\" (UniqueName: \"kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk\") pod \"payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.872891 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.872862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:53.955671 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:53.955604    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" path="/var/lib/kubelet/pods/5b89f7e6-1e75-4be6-8873-01210381878c/volumes"
Sep 10 21:36:54.140548 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.140521    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:36:54.143031 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:36:54.143003    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3332dbb5_87ef_4611_8e73_5416fcb6eea0.slice/crio-b5af199302ce98ec9e4db9cb11a545e5f1c8d2d1e2405a546f45b3a4cb759f3b WatchSource:0}: Error finding container b5af199302ce98ec9e4db9cb11a545e5f1c8d2d1e2405a546f45b3a4cb759f3b: Status 404 returned error can't find the container with id b5af199302ce98ec9e4db9cb11a545e5f1c8d2d1e2405a546f45b3a4cb759f3b
Sep 10 21:36:54.328100 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.328070    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:36:54.346238 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.346211    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" event={"ID":"3332dbb5-87ef-4611-8e73-5416fcb6eea0","Type":"ContainerStarted","Data":"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"}
Sep 10 21:36:54.346395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.346260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" event={"ID":"3332dbb5-87ef-4611-8e73-5416fcb6eea0","Type":"ContainerStarted","Data":"b5af199302ce98ec9e4db9cb11a545e5f1c8d2d1e2405a546f45b3a4cb759f3b"}
Sep 10 21:36:54.346395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.346291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:36:54.363724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:36:54.363688    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" podStartSLOduration=1.363672756 podStartE2EDuration="1.363672756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:36:54.361918215 +0000 UTC m=+1050.856741753" watchObservedRunningTime="2026-09-10 21:36:54.363672756 +0000 UTC m=+1050.858496338"
Sep 10 21:37:01.644164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:01.644129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:02.181805 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.181761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:02.208462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.208433    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:37:02.208685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.208641    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" podUID="5f991922-5049-4efd-aa34-4ff1bfb80706" containerName="payload-processing" containerID="cri-o://b06faef1b65a69be9691d396c136c0da76ee688c2f8fe125e17e916950dbf6db" gracePeriod=30
Sep 10 21:37:02.214920 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.214896    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:37:02.215132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.215099    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" podUID="a5409052-7805-4ef5-ac9b-d0afbaf46349" containerName="payload-pre-processing" containerID="cri-o://55e3d1272b15b63e4a806e7d30a3d31cfdcd8cc089ac14c599a3ee7b68eedc74" gracePeriod=30
Sep 10 21:37:02.253477 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.253434    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:02.316894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.316861    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:02.385822 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.385755    2579 generic.go:356] "Generic (PLEG): container finished" podID="5f991922-5049-4efd-aa34-4ff1bfb80706" containerID="b06faef1b65a69be9691d396c136c0da76ee688c2f8fe125e17e916950dbf6db" exitCode=0
Sep 10 21:37:02.385966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.385844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" event={"ID":"5f991922-5049-4efd-aa34-4ff1bfb80706","Type":"ContainerDied","Data":"b06faef1b65a69be9691d396c136c0da76ee688c2f8fe125e17e916950dbf6db"}
Sep 10 21:37:02.387959 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.387933    2579 generic.go:356] "Generic (PLEG): container finished" podID="a5409052-7805-4ef5-ac9b-d0afbaf46349" containerID="55e3d1272b15b63e4a806e7d30a3d31cfdcd8cc089ac14c599a3ee7b68eedc74" exitCode=0
Sep 10 21:37:02.388066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.387999    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" event={"ID":"a5409052-7805-4ef5-ac9b-d0afbaf46349","Type":"ContainerDied","Data":"55e3d1272b15b63e4a806e7d30a3d31cfdcd8cc089ac14c599a3ee7b68eedc74"}
Sep 10 21:37:02.417405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.417381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:02.467661 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.467635    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:37:02.494733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.494713    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:37:02.567893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.567863    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4bft\" (UniqueName: \"kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft\") pod \"5f991922-5049-4efd-aa34-4ff1bfb80706\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") "
Sep 10 21:37:02.568082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.567917    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume\") pod \"a5409052-7805-4ef5-ac9b-d0afbaf46349\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") "
Sep 10 21:37:02.568082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.567946    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume\") pod \"5f991922-5049-4efd-aa34-4ff1bfb80706\" (UID: \"5f991922-5049-4efd-aa34-4ff1bfb80706\") "
Sep 10 21:37:02.568082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.567970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n7zh\" (UniqueName: \"kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh\") pod \"a5409052-7805-4ef5-ac9b-d0afbaf46349\" (UID: \"a5409052-7805-4ef5-ac9b-d0afbaf46349\") "
Sep 10 21:37:02.568321 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.568297    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f991922-5049-4efd-aa34-4ff1bfb80706" (UID: "5f991922-5049-4efd-aa34-4ff1bfb80706"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:02.568388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.568334    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a5409052-7805-4ef5-ac9b-d0afbaf46349" (UID: "a5409052-7805-4ef5-ac9b-d0afbaf46349"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:02.570212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.570187    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh" (OuterVolumeSpecName: "kube-api-access-8n7zh") pod "a5409052-7805-4ef5-ac9b-d0afbaf46349" (UID: "a5409052-7805-4ef5-ac9b-d0afbaf46349"). InnerVolumeSpecName "kube-api-access-8n7zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:02.570300 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.570242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft" (OuterVolumeSpecName: "kube-api-access-j4bft") pod "5f991922-5049-4efd-aa34-4ff1bfb80706" (UID: "5f991922-5049-4efd-aa34-4ff1bfb80706"). InnerVolumeSpecName "kube-api-access-j4bft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:02.668954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.668914    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4bft\" (UniqueName: \"kubernetes.io/projected/5f991922-5049-4efd-aa34-4ff1bfb80706-kube-api-access-j4bft\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:02.668954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.668949    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5409052-7805-4ef5-ac9b-d0afbaf46349-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:02.668954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.668959    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f991922-5049-4efd-aa34-4ff1bfb80706-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:02.669348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.668969    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n7zh\" (UniqueName: \"kubernetes.io/projected/a5409052-7805-4ef5-ac9b-d0afbaf46349-kube-api-access-8n7zh\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:02.760597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:02.760521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:03.393966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.393932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"
Sep 10 21:37:03.393966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.393951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml" event={"ID":"5f991922-5049-4efd-aa34-4ff1bfb80706","Type":"ContainerDied","Data":"76ea38ba113316812ddb7df7fc7779434b83d043b1b426e276c3652f84592a0d"}
Sep 10 21:37:03.394229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.394015    2579 scope.go:117] "RemoveContainer" containerID="b06faef1b65a69be9691d396c136c0da76ee688c2f8fe125e17e916950dbf6db"
Sep 10 21:37:03.395424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.395406    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"
Sep 10 21:37:03.395518 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.395476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v" event={"ID":"a5409052-7805-4ef5-ac9b-d0afbaf46349","Type":"ContainerDied","Data":"6ff7c41b47466044ac58e1e5b7919dd46bd5de4ebf2c4cbf65162dfe83c2666a"}
Sep 10 21:37:03.402484 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.402469    2579 scope.go:117] "RemoveContainer" containerID="55e3d1272b15b63e4a806e7d30a3d31cfdcd8cc089ac14c599a3ee7b68eedc74"
Sep 10 21:37:03.417614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.417593    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:37:03.421939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.421912    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-92c5c624-7856d44bc7-r8lml"]
Sep 10 21:37:03.431035 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.431017    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:37:03.437165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.437147    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-92c5c624-7d764f698b-nkh8v"]
Sep 10 21:37:03.956353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.956310    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f991922-5049-4efd-aa34-4ff1bfb80706" path="/var/lib/kubelet/pods/5f991922-5049-4efd-aa34-4ff1bfb80706/volumes"
Sep 10 21:37:03.956881 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:03.956860    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5409052-7805-4ef5-ac9b-d0afbaf46349" path="/var/lib/kubelet/pods/a5409052-7805-4ef5-ac9b-d0afbaf46349/volumes"
Sep 10 21:37:05.354016 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:05.353980    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:37:08.505261 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:08.505216    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:08.570267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:08.570238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:08.748031 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:08.747987    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:37:08.748261 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:08.748234    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy" containerID="cri-o://66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2" gracePeriod=30
Sep 10 21:37:08.774330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:08.774307    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:09.150099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:09.150016    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:09.250879 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:09.250851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:09.383968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:09.383929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:13.793271 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:13.793249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:14.000417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.000395    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:37:14.067629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067552    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg7gl\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067610    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067629 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067626    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067646    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067664    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067707    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067761    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.067990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.067810    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket\") pod \"9d0d55cf-8771-4465-9be9-29cf7881a641\" (UID: \"9d0d55cf-8771-4465-9be9-29cf7881a641\") "
Sep 10 21:37:14.068244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.068011    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:14.068244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.068023    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:14.068244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.068034    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:14.068404 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.068245    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:14.068404 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.068321    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:37:14.070137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.070108    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:37:14.070245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.070150    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:14.070365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.070346    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl" (OuterVolumeSpecName: "kube-api-access-tg7gl") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "kube-api-access-tg7gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:37:14.070405 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.070349    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d0d55cf-8771-4465-9be9-29cf7881a641" (UID: "9d0d55cf-8771-4465-9be9-29cf7881a641"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:37:14.168673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168639    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168670    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168673 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168679    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168687    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168697    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d0d55cf-8771-4465-9be9-29cf7881a641-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168705    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168713    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168720    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d0d55cf-8771-4465-9be9-29cf7881a641-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.168914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.168731    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg7gl\" (UniqueName: \"kubernetes.io/projected/9d0d55cf-8771-4465-9be9-29cf7881a641-kube-api-access-tg7gl\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:37:14.449375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.449282    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerID="66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2" exitCode=0
Sep 10 21:37:14.449375 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.449330    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"
Sep 10 21:37:14.449611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.449369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerDied","Data":"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"}
Sep 10 21:37:14.449611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.449435    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q" event={"ID":"9d0d55cf-8771-4465-9be9-29cf7881a641","Type":"ContainerDied","Data":"b0fbb4ce3cc957a2ae43baa65694944357f58b92281adb7501d3f1fa84ab5615"}
Sep 10 21:37:14.449611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.449468    2579 scope.go:117] "RemoveContainer" containerID="66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"
Sep 10 21:37:14.460187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.460141    2579 scope.go:117] "RemoveContainer" containerID="411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"
Sep 10 21:37:14.468359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.468345    2579 scope.go:117] "RemoveContainer" containerID="66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"
Sep 10 21:37:14.468646 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:37:14.468632    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2\": container with ID starting with 66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2 not found: ID does not exist" containerID="66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"
Sep 10 21:37:14.468699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.468653    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2"} err="failed to get container status \"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2\": rpc error: code = NotFound desc = could not find container \"66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2\": container with ID starting with 66841c8b9f40bf3f4a2c28e534538a57bfd76df7de5c21e4ad828890f488eed2 not found: ID does not exist"
Sep 10 21:37:14.468699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.468667    2579 scope.go:117] "RemoveContainer" containerID="411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"
Sep 10 21:37:14.468936 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:37:14.468915    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05\": container with ID starting with 411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05 not found: ID does not exist" containerID="411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"
Sep 10 21:37:14.469032 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.468938    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05"} err="failed to get container status \"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05\": rpc error: code = NotFound desc = could not find container \"411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05\": container with ID starting with 411433723a5375ad7e1480588c66a675567527b65cef3ec96c5c931afb310d05 not found: ID does not exist"
Sep 10 21:37:14.471871 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.471849    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:37:14.478786 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:14.478749    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-92c5c624-openshift-default-7b6f679d56-m2s2q"]
Sep 10 21:37:15.501476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.501404    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:15.568201 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.568155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:15.658872 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.658847    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:15.754714 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.754625    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:15.922062 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.922028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:15.955063 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:15.955034    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" path="/var/lib/kubelet/pods/9d0d55cf-8771-4465-9be9-29cf7881a641/volumes"
Sep 10 21:37:16.688021 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:16.687976    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:17.052993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.052963    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:17.335864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.335792    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:37:17.336097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336086    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:17.336140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336100    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:17.336140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336108    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f991922-5049-4efd-aa34-4ff1bfb80706" containerName="payload-processing"
Sep 10 21:37:17.336140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336113    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f991922-5049-4efd-aa34-4ff1bfb80706" containerName="payload-processing"
Sep 10 21:37:17.336140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336131    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:17.336140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336137    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336144    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336149    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89f7e6-1e75-4be6-8873-01210381878c" containerName="istio-proxy"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336155    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5409052-7805-4ef5-ac9b-d0afbaf46349" containerName="payload-pre-processing"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336160    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5409052-7805-4ef5-ac9b-d0afbaf46349" containerName="payload-pre-processing"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336205    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336212    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5409052-7805-4ef5-ac9b-d0afbaf46349" containerName="payload-pre-processing"
Sep 10 21:37:17.336298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.336220    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f991922-5049-4efd-aa34-4ff1bfb80706" containerName="payload-processing"
Sep 10 21:37:17.346610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.346585    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.349032 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.348977    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-67d5e2-openshift-default-dockercfg-m6mhj\""
Sep 10 21:37:17.351051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.351028    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:37:17.372177 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.372155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:17.393958 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.393927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.393967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394119    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcljh\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394225    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.394385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.394239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.494816 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.494816 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494820    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494927    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.494995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcljh\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495230    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495419 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495252    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495537 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495497    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.495617 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495589    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.496010 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.495986    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.497400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.497370    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.497796 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.497761    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.503246 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.503222    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.503694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.503675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcljh\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh\") pod \"e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") " pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.660472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.660387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:17.935803 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.935655    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:37:17.938104 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:17.938073    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b3b44c_20fa_47dd_bffc_f03f1328242e.slice/crio-6d3ae5f516618a2fb134372220c095cc4c23df7c406eed176ec49a398a12926c WatchSource:0}: Error finding container 6d3ae5f516618a2fb134372220c095cc4c23df7c406eed176ec49a398a12926c: Status 404 returned error can't find the container with id 6d3ae5f516618a2fb134372220c095cc4c23df7c406eed176ec49a398a12926c
Sep 10 21:37:17.940231 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.940198    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:17.940345 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.940259    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:17.940345 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.940286    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:17.950944 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:17.950926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:18.476954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:18.476912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerStarted","Data":"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"}
Sep 10 21:37:18.477149 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:18.476974    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerStarted","Data":"6d3ae5f516618a2fb134372220c095cc4c23df7c406eed176ec49a398a12926c"}
Sep 10 21:37:18.497793 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:18.497727    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podStartSLOduration=1.497711238 podStartE2EDuration="1.497711238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:18.496557199 +0000 UTC m=+1074.991380751" watchObservedRunningTime="2026-09-10 21:37:18.497711238 +0000 UTC m=+1074.992534778"
Sep 10 21:37:18.661432 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:18.661399    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:18.666437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:18.666413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:19.481284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:19.481247    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:19.482401 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:19.482383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:20.458820 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:20.458763    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:20.556150 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:20.556119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:21.765472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:21.765440    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:21.839749 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:21.839718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.160911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.160830    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.261840 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.261800    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.326159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.326126    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:37:22.326541 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.326524    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d0d55cf-8771-4465-9be9-29cf7881a641" containerName="istio-proxy"
Sep 10 21:37:22.329283 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.329262    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.331665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.331643    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-67d5e2\""
Sep 10 21:37:22.331797 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.331718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-67d5e2-dockercfg-kv56w\""
Sep 10 21:37:22.342247 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.342223    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:37:22.352521 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.352494    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.358698 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.358678    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:37:22.361720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.361707    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.370718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.370695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:37:22.441051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.440898    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.441051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.440948    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptwzb\" (UniqueName: \"kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.441051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.441011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qp2c\" (UniqueName: \"kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.441051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.441057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.517018 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.516988    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.542176 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.542284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542183    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptwzb\" (UniqueName: \"kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.542284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qp2c\" (UniqueName: \"kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.542284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.542822 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.542989 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.542971    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.550893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.550869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptwzb\" (UniqueName: \"kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb\") pod \"payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.551608 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.551593    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qp2c\" (UniqueName: \"kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c\") pod \"payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.584108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.584088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:22.639526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.639498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:22.672697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.672669    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:22.909542 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.909510    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:37:22.912292 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:22.912269    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6049be15_1e81_4fbe_a677_db9c2428eead.slice/crio-93a6070f3bf3d89eb32406629d77600a7dc842f87e0f953471d8884ce2d7d7e5 WatchSource:0}: Error finding container 93a6070f3bf3d89eb32406629d77600a7dc842f87e0f953471d8884ce2d7d7e5: Status 404 returned error can't find the container with id 93a6070f3bf3d89eb32406629d77600a7dc842f87e0f953471d8884ce2d7d7e5
Sep 10 21:37:22.952305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:22.952285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:37:22.954169 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:22.954136    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24d9c529_cab8_48e4_b1ea_3c910fe6a579.slice/crio-64c20cfa6b618d1a0516c4f6339a7b74340cb0d6e1d222099c6cd0ee01f2a6cc WatchSource:0}: Error finding container 64c20cfa6b618d1a0516c4f6339a7b74340cb0d6e1d222099c6cd0ee01f2a6cc: Status 404 returned error can't find the container with id 64c20cfa6b618d1a0516c4f6339a7b74340cb0d6e1d222099c6cd0ee01f2a6cc
Sep 10 21:37:23.462801 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.462739    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:23.500821 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.500770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" event={"ID":"6049be15-1e81-4fbe-a677-db9c2428eead","Type":"ContainerStarted","Data":"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"}
Sep 10 21:37:23.500994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.500965    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:23.501104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.501004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" event={"ID":"6049be15-1e81-4fbe-a677-db9c2428eead","Type":"ContainerStarted","Data":"93a6070f3bf3d89eb32406629d77600a7dc842f87e0f953471d8884ce2d7d7e5"}
Sep 10 21:37:23.502487 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.502465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" event={"ID":"24d9c529-cab8-48e4-b1ea-3c910fe6a579","Type":"ContainerStarted","Data":"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"}
Sep 10 21:37:23.502622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.502515    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" event={"ID":"24d9c529-cab8-48e4-b1ea-3c910fe6a579","Type":"ContainerStarted","Data":"64c20cfa6b618d1a0516c4f6339a7b74340cb0d6e1d222099c6cd0ee01f2a6cc"}
Sep 10 21:37:23.502622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.502606    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:23.516206 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.516160    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" podStartSLOduration=1.516145866 podStartE2EDuration="1.516145866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:23.51503515 +0000 UTC m=+1080.009858689" watchObservedRunningTime="2026-09-10 21:37:23.516145866 +0000 UTC m=+1080.010969404"
Sep 10 21:37:23.530167 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.530126    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" podStartSLOduration=1.530110934 podStartE2EDuration="1.530110934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:23.52920916 +0000 UTC m=+1080.024032700" watchObservedRunningTime="2026-09-10 21:37:23.530110934 +0000 UTC m=+1080.024934469"
Sep 10 21:37:23.749015 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.748940    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:37:23.752729 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.752703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.754843 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.754817    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-4b45c2-openshift-default-dockercfg-s267s\""
Sep 10 21:37:23.767619 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.767577    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:37:23.777011 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.776990    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:23.854388 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.854250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.854657 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.854634    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.854844 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.854824    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.854970 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.854955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.855104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.855089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.855653 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.855629    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.855833 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.855808    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.855990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.855973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xr2\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.856117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.856104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.956554 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.956554 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956578    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xr2\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956764    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.956809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.957171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957466 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.957437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957520 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.957477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.957733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.957674    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.958367 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.958344    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.959208 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.959181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.959751 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.959727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.965514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.965492    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:23.965985 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:23.965966    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xr2\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2\") pod \"e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") " pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:24.046633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.046605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:24.066600 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.066580    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-4b45c2-openshift-default-dockercfg-s267s\""
Sep 10 21:37:24.075155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.075130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:24.357116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.357084    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:37:24.359220 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:24.359196    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45ec884_1a31_4443_a627_b729d3752af0.slice/crio-997da1947f2b2eb8eff1cc6e9178f3655630cb3d5a30599f9fef9c6a796f436b WatchSource:0}: Error finding container 997da1947f2b2eb8eff1cc6e9178f3655630cb3d5a30599f9fef9c6a796f436b: Status 404 returned error can't find the container with id 997da1947f2b2eb8eff1cc6e9178f3655630cb3d5a30599f9fef9c6a796f436b
Sep 10 21:37:24.361498 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.361466    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:24.361589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.361534    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:24.361589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.361562    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:24.510315 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.510275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerStarted","Data":"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"}
Sep 10 21:37:24.510498 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.510343    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerStarted","Data":"997da1947f2b2eb8eff1cc6e9178f3655630cb3d5a30599f9fef9c6a796f436b"}
Sep 10 21:37:24.527674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:24.527615    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podStartSLOduration=1.5275935330000001 podStartE2EDuration="1.527593533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:24.527231679 +0000 UTC m=+1081.022055218" watchObservedRunningTime="2026-09-10 21:37:24.527593533 +0000 UTC m=+1081.022417072"
Sep 10 21:37:25.075826 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:25.075761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:25.081010 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:25.080983    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:25.516103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:25.516080    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:25.517056 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:25.517037    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:26.490180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:26.490150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:26.591313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:26.591287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:27.636218 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:27.636192    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:27.705806 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:27.705728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:27.998492 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:27.998407    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:28.216899 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:28.216864    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:28.318957 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:28.318926    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:34.512234 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:34.512198    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:37:34.512600 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:34.512290    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:37:39.615191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.615159    2579 generic.go:356] "Generic (PLEG): container finished" podID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerID="f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a" exitCode=0
Sep 10 21:37:39.615733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.615210    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerDied","Data":"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"}
Sep 10 21:37:39.615733 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.615690    2579 scope.go:117] "RemoveContainer" containerID="f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"
Sep 10 21:37:39.616887 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.616848    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:39.617004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.616922    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:39.617004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:39.616963    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:40.623165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:40.623129    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerStarted","Data":"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"}
Sep 10 21:37:40.661433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:40.661374    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:40.661743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:40.661719    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:40.661870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:40.661802    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:41.557985 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:41.557941    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:37:41.558180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:41.558010    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:37:41.661607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:41.661570    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:41.662131 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:41.661638    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:42.598229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.598199    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:42.635000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.634967    2579 generic.go:356] "Generic (PLEG): container finished" podID="d45ec884-1a31-4443-a627-b729d3752af0" containerID="55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a" exitCode=0
Sep 10 21:37:42.635159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.635047    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerDied","Data":"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"}
Sep 10 21:37:42.635556 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.635529    2579 scope.go:117] "RemoveContainer" containerID="55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"
Sep 10 21:37:42.637703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.637338    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:42.637703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.637434    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:42.637703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.637480    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:37:42.662207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.662183    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:42.662789 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:42.662662    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:43.649088 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:43.649046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerStarted","Data":"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"}
Sep 10 21:37:43.661837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:43.661754    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:43.662019 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:43.661877    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:44.076159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:44.076099    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:44.076973 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:44.076931    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:37:44.077198 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:44.077155    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:37:44.661876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:44.661842    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:44.662166 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:44.662136    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:45.076409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:45.076363    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:37:45.076983 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:45.076434    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:37:45.453682 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:45.453559    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:45.661597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:45.661556    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 10 21:37:45.661837 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:45.661637    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 10 21:37:46.076517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:46.076481    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:37:46.076948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:46.076530    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:37:46.665866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:46.665830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:46.666298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:46.666277    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:46.667118 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:46.667099    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:37:47.076212 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:47.076171    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:37:47.076404 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:47.076243    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:37:48.075915 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:48.075881    2579 patch_prober.go:28] interesting pod/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Sep 10 21:37:48.076296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:48.075935    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Sep 10 21:37:48.726992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:48.726959    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:49.080296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:49.080268    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:49.080665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:49.080632    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:49.081393 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:49.081373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:37:51.141377 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:51.141350    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:51.518494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:51.518466    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:51.843251 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:51.843173    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:52.150320 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:52.150249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:52.517996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:52.517969    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:54.964114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:54.964081    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:55.062764 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:55.062728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:56.116140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.116113    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:56.216557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.216528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:56.679462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.679425    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:56.777857 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.777825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:37:56.791607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.791582    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:37:56.795000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.794983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.797213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.797178    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-b4c9bb\""
Sep 10 21:37:56.797400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.797380    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-b4c9bb-dockercfg-bnd7d\""
Sep 10 21:37:56.808930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.808891    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:37:56.811199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.811178    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:37:56.814723 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.814706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.825005 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.824984    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:37:56.859224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.859192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.859224 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.859227    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llqs\" (UniqueName: \"kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.859455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.859309    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.859455 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.859438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k85b\" (UniqueName: \"kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.960313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.960233    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k85b\" (UniqueName: \"kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.960313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.960289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.960512 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.960315    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llqs\" (UniqueName: \"kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.960512 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.960348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.961289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.961264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.961365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.961264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:56.972307 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.972276    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k85b\" (UniqueName: \"kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b\") pod \"payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:56.972531 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:56.972513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llqs\" (UniqueName: \"kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs\") pod \"payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:57.105649 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.105605    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:57.124878 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.124848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:57.377421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.377397    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:37:57.380044 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:57.380017    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a74c279_1c1c_4cc9_82c7_6fc336fb62eb.slice/crio-3c0251a3523a7cf99d5df053dce40584cec9c09a2cc1a1f862ec6b6644c5e39a WatchSource:0}: Error finding container 3c0251a3523a7cf99d5df053dce40584cec9c09a2cc1a1f862ec6b6644c5e39a: Status 404 returned error can't find the container with id 3c0251a3523a7cf99d5df053dce40584cec9c09a2cc1a1f862ec6b6644c5e39a
Sep 10 21:37:57.405325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.405301    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:37:57.408886 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:37:57.408858    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95429d86_01cc_410b_a4c1_cd857026c15b.slice/crio-5421ea37540272c52b6c2bc77c439142aefda5f0d60e2b445ceec16544924f4a WatchSource:0}: Error finding container 5421ea37540272c52b6c2bc77c439142aefda5f0d60e2b445ceec16544924f4a: Status 404 returned error can't find the container with id 5421ea37540272c52b6c2bc77c439142aefda5f0d60e2b445ceec16544924f4a
Sep 10 21:37:57.725130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.725038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" event={"ID":"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb","Type":"ContainerStarted","Data":"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"}
Sep 10 21:37:57.725130 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.725107    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:37:57.725365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.725135    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" event={"ID":"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb","Type":"ContainerStarted","Data":"3c0251a3523a7cf99d5df053dce40584cec9c09a2cc1a1f862ec6b6644c5e39a"}
Sep 10 21:37:57.726742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.726713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" event={"ID":"95429d86-01cc-410b-a4c1-cd857026c15b","Type":"ContainerStarted","Data":"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"}
Sep 10 21:37:57.726908 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.726757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" event={"ID":"95429d86-01cc-410b-a4c1-cd857026c15b","Type":"ContainerStarted","Data":"5421ea37540272c52b6c2bc77c439142aefda5f0d60e2b445ceec16544924f4a"}
Sep 10 21:37:57.726908 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.726814    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:37:57.741001 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.740953    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" podStartSLOduration=1.740935178 podStartE2EDuration="1.740935178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:57.74029387 +0000 UTC m=+1114.235117411" watchObservedRunningTime="2026-09-10 21:37:57.740935178 +0000 UTC m=+1114.235758717"
Sep 10 21:37:57.757126 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:37:57.757079    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" podStartSLOduration=1.757065786 podStartE2EDuration="1.757065786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:37:57.755755495 +0000 UTC m=+1114.250579035" watchObservedRunningTime="2026-09-10 21:37:57.757065786 +0000 UTC m=+1114.251889323"
Sep 10 21:38:08.733835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:08.733792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:38:08.734368 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:08.733899    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:38:19.933236 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:19.933203    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:31.483747 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.483712    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:31.754792 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.754688    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:38:31.758511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.758486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.760701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.760678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8c245620-openshift-default-dockercfg-nnf9n\""
Sep 10 21:38:31.770618 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.770596    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:38:31.781891 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.781866    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:31.873340 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873709 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873709 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873709 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873625    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.873709 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.873670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w5xs\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.952864 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.952825    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:31.974339 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974308    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w5xs\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974353    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974444    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974490 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974862 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974622    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974862 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974862 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974862 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974727    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.974862 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.975137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.974980    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.975137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.975021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.975387 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.975365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.976941 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.976922    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.977083 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.977065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.981804 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.981765    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:31.982171 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:31.982151    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w5xs\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs\") pod \"e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") " pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:32.070066 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.070032    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:32.345112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.345089    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:38:32.347367 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:38:32.347336    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c5ac46d_b07e_468e_8d0c_44c3b1d7cad1.slice/crio-5fa2fd4639a6f2597d6674fabb6f777f27e7ae4c9522b204e941f16e0a7f1bb4 WatchSource:0}: Error finding container 5fa2fd4639a6f2597d6674fabb6f777f27e7ae4c9522b204e941f16e0a7f1bb4: Status 404 returned error can't find the container with id 5fa2fd4639a6f2597d6674fabb6f777f27e7ae4c9522b204e941f16e0a7f1bb4
Sep 10 21:38:32.352820 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.352772    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:32.352892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.352862    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:32.352929 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.352891    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:32.903453 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.903415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerStarted","Data":"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"}
Sep 10 21:38:32.903852 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.903472    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerStarted","Data":"5fa2fd4639a6f2597d6674fabb6f777f27e7ae4c9522b204e941f16e0a7f1bb4"}
Sep 10 21:38:32.923240 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:32.923196    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podStartSLOduration=1.923180995 podStartE2EDuration="1.923180995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:38:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:38:32.921016063 +0000 UTC m=+1149.415839601" watchObservedRunningTime="2026-09-10 21:38:32.923180995 +0000 UTC m=+1149.418004532"
Sep 10 21:38:33.070180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:33.070143    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:33.075314 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:33.075288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:33.908853 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:33.908808    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:33.909699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:33.909677    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:34.812590 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:34.812558    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:34.909341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:34.909297    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:35.952350 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:35.952315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:36.030384 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:36.030352    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:36.130319 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:36.130282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:36.531685 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:36.531657    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:36.632284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:36.632228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:38:56.014948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.014916    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerID="04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8" exitCode=0
Sep 10 21:38:56.015356 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.014990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerDied","Data":"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"}
Sep 10 21:38:56.015414 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.015359    2579 scope.go:117] "RemoveContainer" containerID="04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"
Sep 10 21:38:56.016421 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.016384    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:56.016549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.016475    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:56.016549 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:56.016527    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:38:57.021756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:57.021717    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerStarted","Data":"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"}
Sep 10 21:38:57.070197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:57.070156    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:38:57.070383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:57.070345    2579 patch_prober.go:28] interesting pod/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 10 21:38:57.070444 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:57.070415    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 10 21:38:58.071199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:58.071164    2579 patch_prober.go:28] interesting pod/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 10 21:38:58.071585 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:58.071241    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 10 21:38:59.071128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:59.071092    2579 patch_prober.go:28] interesting pod/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 10 21:38:59.071368 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:59.071166    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 10 21:38:59.796555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:38:59.796521    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:00.071036 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:00.070942    2579 patch_prober.go:28] interesting pod/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 10 21:39:00.071036 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:00.071004    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 10 21:39:01.071317 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:01.071284    2579 patch_prober.go:28] interesting pod/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 10 21:39:01.071835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:01.071347    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 10 21:39:02.140785 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:02.140753    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:39:02.141246 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:02.141216    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:39:02.141994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:02.141975    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:39:02.835726 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:02.835699    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:16.737601 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:16.737563    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:18.047926 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:18.047889    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:24.014979 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:24.014950    2579 scope.go:117] "RemoveContainer" containerID="23d86b0c2a613624e9385f8fb148cd77ca999673743edd0f36e0e9c4cfe70dff"
Sep 10 21:39:24.023096 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:24.023077    2579 scope.go:117] "RemoveContainer" containerID="7c85b5bba579b63d18501b14c60e7300c0f853c1d173765015c9440e56f09b2a"
Sep 10 21:39:59.720704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.720670    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"]
Sep 10 21:39:59.723768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.723753    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:39:59.726214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.726181    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:39:59.726373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.726356    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:39:59.736385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.736364    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"]
Sep 10 21:39:59.858744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.858710    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8c245620-g8f67\" (UID: \"e602b6ce-cea2-437d-9ad0-95f0b048f1f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:39:59.922058 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.922028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:39:59.959671 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.959640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8c245620-g8f67\" (UID: \"e602b6ce-cea2-437d-9ad0-95f0b048f1f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:39:59.960180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:39:59.960163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8c245620-g8f67\" (UID: \"e602b6ce-cea2-437d-9ad0-95f0b048f1f9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:40:00.033689 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:00.033658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:40:00.302753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:00.302731    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"]
Sep 10 21:40:00.305030 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:40:00.304992    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode602b6ce_cea2_437d_9ad0_95f0b048f1f9.slice/crio-de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa WatchSource:0}: Error finding container de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa: Status 404 returned error can't find the container with id de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa
Sep 10 21:40:00.340727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:00.340690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67" event={"ID":"e602b6ce-cea2-437d-9ad0-95f0b048f1f9","Type":"ContainerStarted","Data":"de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa"}
Sep 10 21:40:01.347743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:01.347645    2579 generic.go:356] "Generic (PLEG): container finished" podID="e602b6ce-cea2-437d-9ad0-95f0b048f1f9" containerID="65a5b52ff76363879ed8a66209d87f30c4284d85d3b5d0d9b44825c864a8a8cd" exitCode=0
Sep 10 21:40:01.347743 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:01.347730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67" event={"ID":"e602b6ce-cea2-437d-9ad0-95f0b048f1f9","Type":"ContainerDied","Data":"65a5b52ff76363879ed8a66209d87f30c4284d85d3b5d0d9b44825c864a8a8cd"}
Sep 10 21:40:02.477298 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:02.477274    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:40:02.480345 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:02.480326    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca\") pod \"e602b6ce-cea2-437d-9ad0-95f0b048f1f9\" (UID: \"e602b6ce-cea2-437d-9ad0-95f0b048f1f9\") "
Sep 10 21:40:02.480675 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:02.480654    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e602b6ce-cea2-437d-9ad0-95f0b048f1f9" (UID: "e602b6ce-cea2-437d-9ad0-95f0b048f1f9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:40:02.581214 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:02.581189    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e602b6ce-cea2-437d-9ad0-95f0b048f1f9-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:40:03.359136 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:03.359105    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"
Sep 10 21:40:03.359305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:03.359112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67" event={"ID":"e602b6ce-cea2-437d-9ad0-95f0b048f1f9","Type":"ContainerDied","Data":"de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa"}
Sep 10 21:40:03.359305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:03.359247    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de99d8a6b5bb272f86e55bc84f8979eb1e3f14e073a9322123555208e0a096aa"
Sep 10 21:40:15.274003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.273972    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:15.374199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.374152    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:15.450972 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.450905    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"]
Sep 10 21:40:15.453289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.453268    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8c245620-g8f67"]
Sep 10 21:40:15.453447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.453392    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:15.515437 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.515390    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:15.955557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:15.955526    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e602b6ce-cea2-437d-9ad0-95f0b048f1f9" path="/var/lib/kubelet/pods/e602b6ce-cea2-437d-9ad0-95f0b048f1f9/volumes"
Sep 10 21:40:18.591840 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:18.591809    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:23.615126 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:23.615088    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:24.043770 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:24.043741    2579 scope.go:117] "RemoveContainer" containerID="b23053b306ea939cc1180f3ee847a54381276bf0c971a0c6a6e028d1e737156f"
Sep 10 21:40:24.051832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:24.051812    2579 scope.go:117] "RemoveContainer" containerID="a212d9457dfd69951ce000f65a957fef19b0152c16e1cfc156a8955809a79caf"
Sep 10 21:40:39.426218 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:39.426183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:39.525362 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:39.525323    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:47.871497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:47.871459    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:47.928170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:47.928142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:40:59.828327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:40:59.828291    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:01.076990 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:01.076945    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:06.644727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:06.644693    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:06.716652 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:06.716615    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:06.886422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:06.886390    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:41:06.886618 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:06.886597    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy" containerID="cri-o://926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8" gracePeriod=30
Sep 10 21:41:06.908343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:06.908274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:08.202494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:08.202462    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:08.320831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:08.320799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:08.440439 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:08.440410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:09.072695 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:09.072661    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:09.190649 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:09.190619    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:10.511414 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:10.511381    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:12.142117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.142096    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:41:12.271292 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271267    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271303    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271338    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271401    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271434    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271521    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271556    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:12.271875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271677    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w5xs\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs\") pod \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\" (UID: \"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1\") "
Sep 10 21:41:12.271875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271687    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:12.271875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271765    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:12.271875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271804    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.271867    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.272003    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.272021    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.272035    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.272050    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.272080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.272067    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.273660 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.273631    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:41:12.273840 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.273719    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs" (OuterVolumeSpecName: "kube-api-access-7w5xs") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "kube-api-access-7w5xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:41:12.274165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.274136    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:41:12.274248 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.274168    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" (UID: "0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:41:12.373275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.373253    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.373275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.373273    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w5xs\" (UniqueName: \"kubernetes.io/projected/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-kube-api-access-7w5xs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.373410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.373283    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.373410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.373291    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:41:12.691830 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.691730    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerID="926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8" exitCode=0
Sep 10 21:41:12.691830 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.691793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerDied","Data":"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"}
Sep 10 21:41:12.691830 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.691801    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"
Sep 10 21:41:12.692077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.691848    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh" event={"ID":"0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1","Type":"ContainerDied","Data":"5fa2fd4639a6f2597d6674fabb6f777f27e7ae4c9522b204e941f16e0a7f1bb4"}
Sep 10 21:41:12.692077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.691873    2579 scope.go:117] "RemoveContainer" containerID="926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"
Sep 10 21:41:12.700875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.700858    2579 scope.go:117] "RemoveContainer" containerID="04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"
Sep 10 21:41:12.708577 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.708561    2579 scope.go:117] "RemoveContainer" containerID="926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"
Sep 10 21:41:12.708870 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:41:12.708850    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8\": container with ID starting with 926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8 not found: ID does not exist" containerID="926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"
Sep 10 21:41:12.708922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.708880    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8"} err="failed to get container status \"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8\": rpc error: code = NotFound desc = could not find container \"926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8\": container with ID starting with 926ee700b35a8e753fe4ba8dc0da996c85cf8df0f024a2df0759e4a9e3ff6be8 not found: ID does not exist"
Sep 10 21:41:12.708922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.708898    2579 scope.go:117] "RemoveContainer" containerID="04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"
Sep 10 21:41:12.709154 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:41:12.709139    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8\": container with ID starting with 04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8 not found: ID does not exist" containerID="04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"
Sep 10 21:41:12.709194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.709159    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8"} err="failed to get container status \"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8\": rpc error: code = NotFound desc = could not find container \"04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8\": container with ID starting with 04428400a3d6789a407c8b660426302b0655e8bd0ad49db0e60b6738f88579e8 not found: ID does not exist"
Sep 10 21:41:12.711658 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.711640    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:41:12.716432 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:12.716415    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8c245620-openshift-default-86d589c6cb-vj6mh"]
Sep 10 21:41:13.955616 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:13.955582    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" path="/var/lib/kubelet/pods/0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1/volumes"
Sep 10 21:41:16.384185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:16.384149    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:16.446745 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:16.446718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:17.102202 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:17.102169    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:21.437978 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:21.437890    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:22.539170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:22.539141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:23.101227 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:23.101191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:23.199919 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:23.199885    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:23.301450 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:23.301416    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:24.073573 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:24.073543    2579 scope.go:117] "RemoveContainer" containerID="68e32ecf5c6c866322aeec9c0b3b1a9ad5eff63b8392d10cbf8217e2ad5a33e6"
Sep 10 21:41:27.626134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:27.626086    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:31.543955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:31.543925    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:33.774753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:33.774725    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:34.012484 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:34.012452    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:54.048867 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:54.048747    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:41:54.960002 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:41:54.959971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:01.221069 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.221037    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:01.318548 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.318504    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:01.584573 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.584543    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:01.960727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.960652    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:42:01.961017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961003    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e602b6ce-cea2-437d-9ad0-95f0b048f1f9" containerName="revoke-keys"
Sep 10 21:42:01.961087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961020    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e602b6ce-cea2-437d-9ad0-95f0b048f1f9" containerName="revoke-keys"
Sep 10 21:42:01.961087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961038    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961044    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961060    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961087 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961065    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961117    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961127    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c5ac46d-b07e-468e-8d0c-44c3b1d7cad1" containerName="istio-proxy"
Sep 10 21:42:01.961312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.961135    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e602b6ce-cea2-437d-9ad0-95f0b048f1f9" containerName="revoke-keys"
Sep 10 21:42:01.964098 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.964079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:01.966328 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.966308    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-93f046-openshift-default-dockercfg-vkfkp\""
Sep 10 21:42:01.971887 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.971866    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:42:01.985121 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:01.985093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:02.081846 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.081814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.081863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.081902    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.081924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.082016    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4pb2\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.082060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.082099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.082117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.082213 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.082186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182583 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182589    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182709    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.182748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182806    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4pb2\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.182851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.183032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.183140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.183224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.183366    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.183909 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.183883    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.185102 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.185081    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.185211 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.185181    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.191117 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.191093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.191302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.191281    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4pb2\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2\") pod \"e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") " pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.277175 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.277140    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:02.466445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.466413    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:02.557614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.557591    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:42:02.562121 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:42:02.562084    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b801ccc_499a_43d2_b92f_2e0c76321c5b.slice/crio-efe4ca9a154d7608df1f3d1db23029fc045aed052705b612c8da124ee79164f2 WatchSource:0}: Error finding container efe4ca9a154d7608df1f3d1db23029fc045aed052705b612c8da124ee79164f2: Status 404 returned error can't find the container with id efe4ca9a154d7608df1f3d1db23029fc045aed052705b612c8da124ee79164f2
Sep 10 21:42:02.564097 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.564061    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:02.564192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.564131    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:02.564192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.564165    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:02.921791 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.921692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerStarted","Data":"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"}
Sep 10 21:42:02.921791 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.921753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerStarted","Data":"efe4ca9a154d7608df1f3d1db23029fc045aed052705b612c8da124ee79164f2"}
Sep 10 21:42:02.942315 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:02.942272    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podStartSLOduration=1.94224807 podStartE2EDuration="1.94224807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:42:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:42:02.940305294 +0000 UTC m=+1359.435128844" watchObservedRunningTime="2026-09-10 21:42:02.94224807 +0000 UTC m=+1359.437071608"
Sep 10 21:42:03.277578 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:03.277545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:03.282803 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:03.282765    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:03.926669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:03.926634    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:03.927428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:03.927409    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:04.864080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:04.864030    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:04.960669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:04.960639    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:06.015340 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:06.015299    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:06.113052 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:06.113014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:06.346614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:06.346519    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:06.482463 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:06.482428    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:06.565885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:06.565855    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:14.596877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:14.596844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:24.096893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:24.096861    2579 scope.go:117] "RemoveContainer" containerID="341ca34cc5b74094bc1986192024ef0a6f27c7950b6fbe4b48dec93de949dcc7"
Sep 10 21:42:24.106925 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:24.106901    2579 scope.go:117] "RemoveContainer" containerID="69bd133d24ecd7e8f345a9a61cb367bb5fe878f115177cc96dfc984bc9dd20d4"
Sep 10 21:42:26.033646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.033618    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerID="0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8" exitCode=0
Sep 10 21:42:26.033993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.033687    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerDied","Data":"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"}
Sep 10 21:42:26.034115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.034102    2579 scope.go:117] "RemoveContainer" containerID="0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"
Sep 10 21:42:26.035354 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.035327    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:26.035428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.035397    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:26.035465 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:26.035429    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:27.043222 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:27.043186    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerStarted","Data":"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"}
Sep 10 21:42:27.277489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:27.277453    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:27.277734 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:27.277712    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:27.277817 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:27.277766    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:27.820050 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:27.820019    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:28.278481 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:28.278443    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:28.278984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:28.278502    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:29.278476 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:29.278439    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:29.278669 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:29.278496    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:29.807443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:29.807412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:30.249964 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.249835    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:30.277727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.277694    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:30.277923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.277749    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:30.527060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.526752    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:42:30.529841 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.529815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.532031 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.532012    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a867da-openshift-default-dockercfg-hrjtj\""
Sep 10 21:42:30.564799 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.560076    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:42:30.575855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.575828    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:30.661497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.661451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.661880 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.661842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.662108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.662059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.662284 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.662267    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.662508 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.662489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.662713 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.662697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.662932 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.662890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.663144 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.663104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glr6h\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.663415 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.663397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764067 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764036    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764359    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764436 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764428    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glr6h\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764527 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764519    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764639 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764729 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.764841 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765002 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.764986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.765362    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.765396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765574 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.765453    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.765798    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.765972 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.765809    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.767272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.767254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.767923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.767904    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.772386 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.772363    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.772948 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.772916    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glr6h\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h\") pod \"e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") " pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:30.827085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.827014    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:30.854047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:30.854009    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:31.141445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.141349    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:42:31.165809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.162932    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:31.165809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.163028    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:31.165809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.163074    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:31.278228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.278194    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:31.278394 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.278252    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:31.726343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.726310    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:31.827740 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:31.827711    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:32.074697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.074661    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerStarted","Data":"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"}
Sep 10 21:42:32.074898 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.074716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerStarted","Data":"7aefc997cc48322a13895116ec0692936d37146ca2f1feba365d1fb1821dc3d3"}
Sep 10 21:42:32.092577 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.092522    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podStartSLOduration=2.092502534 podStartE2EDuration="2.092502534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:42:32.091476476 +0000 UTC m=+1388.586300015" watchObservedRunningTime="2026-09-10 21:42:32.092502534 +0000 UTC m=+1388.587326074"
Sep 10 21:42:32.277949 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.277908    2579 patch_prober.go:28] interesting pod/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 10 21:42:32.278132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.277966    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 10 21:42:32.854755 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.854713    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:32.860102 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:32.860072    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:33.080221 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:33.079556    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:33.080489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:33.080471    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:33.282171 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:33.282142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:33.282566 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:33.282548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:33.283260 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:33.283240    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:42:34.007566 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:34.007533    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:34.096742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:34.096714    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.142190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.142098    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.246485 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.246458    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.343285 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.343229    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.626115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.626085    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.702877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.702846    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:35.779330 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.779298    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:42:35.782893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.782877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.785216 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.785190    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a867da\""
Sep 10 21:42:35.785338 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.785225    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a867da-dockercfg-q5gqb\""
Sep 10 21:42:35.793897 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.793869    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:42:35.806191 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.806163    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.806290 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.806235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp4lm\" (UniqueName: \"kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.907063 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.906979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp4lm\" (UniqueName: \"kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.907216 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.907088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.907859 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.907839    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:35.915099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:35.915076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp4lm\" (UniqueName: \"kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm\") pod \"payload-processing-e2e-api-b-a867da-5f459d5595-x59g9\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") " pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:36.094529 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:36.094499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:36.366428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:36.366401    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:42:36.368759 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:42:36.368725    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477733d3_cf98_4fb5_b4f9_98d25084ed96.slice/crio-52f177af479e408950224f8c2636fa408a07e6111c664a6c6da7bafe32df41fe WatchSource:0}: Error finding container 52f177af479e408950224f8c2636fa408a07e6111c664a6c6da7bafe32df41fe: Status 404 returned error can't find the container with id 52f177af479e408950224f8c2636fa408a07e6111c664a6c6da7bafe32df41fe
Sep 10 21:42:37.098856 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:37.098817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" event={"ID":"477733d3-cf98-4fb5-b4f9-98d25084ed96","Type":"ContainerStarted","Data":"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"}
Sep 10 21:42:37.099053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:37.098876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" event={"ID":"477733d3-cf98-4fb5-b4f9-98d25084ed96","Type":"ContainerStarted","Data":"52f177af479e408950224f8c2636fa408a07e6111c664a6c6da7bafe32df41fe"}
Sep 10 21:42:37.099053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:37.098929    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:37.114718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:37.114660    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" podStartSLOduration=2.114642342 podStartE2EDuration="2.114642342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:42:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:42:37.114052875 +0000 UTC m=+1393.608876415" watchObservedRunningTime="2026-09-10 21:42:37.114642342 +0000 UTC m=+1393.609465880"
Sep 10 21:42:43.446273 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:43.446244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:48.105754 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:48.105717    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:42:49.364646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:49.364606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:50.647831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:50.647790    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:54.624267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:54.624237    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:55.531566 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:55.531528    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:56.143438 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:56.143393    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:57.198736 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.198706    2579 generic.go:356] "Generic (PLEG): container finished" podID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerID="9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82" exitCode=0
Sep 10 21:42:57.199120 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.198800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerDied","Data":"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"}
Sep 10 21:42:57.199240 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.199226    2579 scope.go:117] "RemoveContainer" containerID="9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"
Sep 10 21:42:57.200425 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.200399    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:57.200478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.200464    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:57.200513 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:57.200495    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:42:58.205662 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.205626    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerStarted","Data":"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"}
Sep 10 21:42:58.426832 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.426799    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:58.856692 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.856638    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:42:58.856968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.856919    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 21:42:58.857045 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.856980    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 21:42:58.947198 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:58.947161    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:42:59.854536 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:59.854497    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 21:42:59.855164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:42:59.855134    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 21:43:00.855057 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:00.855018    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 21:43:00.855646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:00.855611    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 21:43:01.848875 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.848770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"]
Sep 10 21:43:01.853093 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.853066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:01.854911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.854887    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 21:43:01.855105 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.855080    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 21:43:01.855749 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.855726    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:43:01.855890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.855864    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:43:01.860065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.860045    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"]
Sep 10 21:43:01.979991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:01.979955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93f046-f6g9k\" (UID: \"d7833b87-6222-4ba0-97cb-4da04b9438ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:02.053547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.053514    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:02.080562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.080533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93f046-f6g9k\" (UID: \"d7833b87-6222-4ba0-97cb-4da04b9438ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:02.081180 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.081162    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-93f046-f6g9k\" (UID: \"d7833b87-6222-4ba0-97cb-4da04b9438ff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:02.170645 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.170552    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:02.439543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.439452    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"]
Sep 10 21:43:02.456386 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:43:02.456338    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7833b87_6222_4ba0_97cb_4da04b9438ff.slice/crio-2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b WatchSource:0}: Error finding container 2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b: Status 404 returned error can't find the container with id 2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b
Sep 10 21:43:02.854672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.854631    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 10 21:43:02.854911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:02.854700    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 10 21:43:03.080113 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.080082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:43:03.233707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.233673    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k" event={"ID":"d7833b87-6222-4ba0-97cb-4da04b9438ff","Type":"ContainerStarted","Data":"ef8a6d7d74ef6067e847c867378adac7fe01444db7a2eec95881b86a3de92444"}
Sep 10 21:43:03.233922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.233733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k" event={"ID":"d7833b87-6222-4ba0-97cb-4da04b9438ff","Type":"ContainerStarted","Data":"2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b"}
Sep 10 21:43:03.248111 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.248062    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k" podStartSLOduration=2.248046811 podStartE2EDuration="2.248046811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:43:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:43:03.246619846 +0000 UTC m=+1419.741443387" watchObservedRunningTime="2026-09-10 21:43:03.248046811 +0000 UTC m=+1419.742870349"
Sep 10 21:43:03.858930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.858893    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:43:03.860048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:03.860029    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:43:04.244754 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:04.244660    2579 generic.go:356] "Generic (PLEG): container finished" podID="d7833b87-6222-4ba0-97cb-4da04b9438ff" containerID="ef8a6d7d74ef6067e847c867378adac7fe01444db7a2eec95881b86a3de92444" exitCode=0
Sep 10 21:43:04.244754 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:04.244708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k" event={"ID":"d7833b87-6222-4ba0-97cb-4da04b9438ff","Type":"ContainerDied","Data":"ef8a6d7d74ef6067e847c867378adac7fe01444db7a2eec95881b86a3de92444"}
Sep 10 21:43:05.382304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:05.382279    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:05.514194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:05.514163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca\") pod \"d7833b87-6222-4ba0-97cb-4da04b9438ff\" (UID: \"d7833b87-6222-4ba0-97cb-4da04b9438ff\") "
Sep 10 21:43:05.514540 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:05.514516    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7833b87-6222-4ba0-97cb-4da04b9438ff" (UID: "d7833b87-6222-4ba0-97cb-4da04b9438ff"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:05.615337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:05.615282    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7833b87-6222-4ba0-97cb-4da04b9438ff-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:06.257194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:06.257100    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"
Sep 10 21:43:06.257194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:06.257137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k" event={"ID":"d7833b87-6222-4ba0-97cb-4da04b9438ff","Type":"ContainerDied","Data":"2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b"}
Sep 10 21:43:06.257194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:06.257195    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2505067af73cb806224050c0c8e17d39383abac153085a8299ae55406012c65b"
Sep 10 21:43:17.424708 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.424635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:17.509133 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.509093    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:17.583753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.583724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:17.613494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.613461    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"]
Sep 10 21:43:17.615863 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.615841    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-93f046-f6g9k"]
Sep 10 21:43:17.680813 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.680724    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:17.782008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.781982    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:17.955650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:17.955571    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7833b87-6222-4ba0-97cb-4da04b9438ff" path="/var/lib/kubelet/pods/d7833b87-6222-4ba0-97cb-4da04b9438ff/volumes"
Sep 10 21:43:18.060317 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:18.060288    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:23.752446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:23.752411    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:23.839403 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:23.839373    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:24.007425 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.007334    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:43:24.007647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.007607    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy" containerID="cri-o://391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2" gracePeriod=30
Sep 10 21:43:24.050008 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.049976    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:24.131897 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.131862    2579 scope.go:117] "RemoveContainer" containerID="91bc71a85b2cbf240f90280e882de4b870dccbea6a482109d766f80b42d0d152"
Sep 10 21:43:24.435696 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.435665    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"]
Sep 10 21:43:24.436141 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.436123    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7833b87-6222-4ba0-97cb-4da04b9438ff" containerName="revoke-keys"
Sep 10 21:43:24.436235 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.436144    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7833b87-6222-4ba0-97cb-4da04b9438ff" containerName="revoke-keys"
Sep 10 21:43:24.436295 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.436248    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7833b87-6222-4ba0-97cb-4da04b9438ff" containerName="revoke-keys"
Sep 10 21:43:24.440597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.440574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:24.444488 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.444468    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:43:24.444579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.444469    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:43:24.460758 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.460730    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"]
Sep 10 21:43:24.481663 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.481627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a867da-74qgv\" (UID: \"0dcf88ec-164c-4ad4-be5e-d3e732e8b486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:24.582714 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.582682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a867da-74qgv\" (UID: \"0dcf88ec-164c-4ad4-be5e-d3e732e8b486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:24.583306 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.583285    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a867da-74qgv\" (UID: \"0dcf88ec-164c-4ad4-be5e-d3e732e8b486\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:24.679240 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.679211    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:24.750614 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:24.750520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:25.020871 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:25.020845    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"]
Sep 10 21:43:25.023388 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:43:25.023361    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dcf88ec_164c_4ad4_be5e_d3e732e8b486.slice/crio-3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a WatchSource:0}: Error finding container 3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a: Status 404 returned error can't find the container with id 3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a
Sep 10 21:43:25.353363 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:25.353275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv" event={"ID":"0dcf88ec-164c-4ad4-be5e-d3e732e8b486","Type":"ContainerStarted","Data":"8ebfb66616452bc4a835b90b5228bbc636f101416b57b2385928766c805b37cf"}
Sep 10 21:43:25.353363 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:25.353337    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv" event={"ID":"0dcf88ec-164c-4ad4-be5e-d3e732e8b486","Type":"ContainerStarted","Data":"3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a"}
Sep 10 21:43:25.367582 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:25.367537    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv" podStartSLOduration=1.3675164899999999 podStartE2EDuration="1.36751649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:43:25.3661334 +0000 UTC m=+1441.860956939" watchObservedRunningTime="2026-09-10 21:43:25.36751649 +0000 UTC m=+1441.862340061"
Sep 10 21:43:26.359596 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:26.359565    2579 generic.go:356] "Generic (PLEG): container finished" podID="0dcf88ec-164c-4ad4-be5e-d3e732e8b486" containerID="8ebfb66616452bc4a835b90b5228bbc636f101416b57b2385928766c805b37cf" exitCode=0
Sep 10 21:43:26.360017 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:26.359654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv" event={"ID":"0dcf88ec-164c-4ad4-be5e-d3e732e8b486","Type":"ContainerDied","Data":"8ebfb66616452bc4a835b90b5228bbc636f101416b57b2385928766c805b37cf"}
Sep 10 21:43:27.496810 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:27.496764    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:27.607204 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:27.607169    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca\") pod \"0dcf88ec-164c-4ad4-be5e-d3e732e8b486\" (UID: \"0dcf88ec-164c-4ad4-be5e-d3e732e8b486\") "
Sep 10 21:43:27.607518 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:27.607494    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0dcf88ec-164c-4ad4-be5e-d3e732e8b486" (UID: "0dcf88ec-164c-4ad4-be5e-d3e732e8b486"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:27.707860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:27.707763    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0dcf88ec-164c-4ad4-be5e-d3e732e8b486-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:28.370022 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:28.369992    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"
Sep 10 21:43:28.370197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:28.370024    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv" event={"ID":"0dcf88ec-164c-4ad4-be5e-d3e732e8b486","Type":"ContainerDied","Data":"3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a"}
Sep 10 21:43:28.370197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:28.370072    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ae50d810c61012e5974975f8b6d0613098b3e907cfad1957277296082a7191a"
Sep 10 21:43:29.248293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.248266    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:43:29.324676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324571    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.324676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324638    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.324676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324655    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.324676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324673    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4pb2\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324737    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324838    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324860    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324884    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert\") pod \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\" (UID: \"5b801ccc-499a-43d2-b92f-2e0c76321c5b\") "
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324939    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:29.325047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.324954    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325146    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325300    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325316    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325325    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325337    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.325434 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.325409    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:29.327255 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.327231    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:29.327400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.327372    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:43:29.327444 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.327425    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2" (OuterVolumeSpecName: "kube-api-access-r4pb2") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "kube-api-access-r4pb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:43:29.327480 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.327437    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5b801ccc-499a-43d2-b92f-2e0c76321c5b" (UID: "5b801ccc-499a-43d2-b92f-2e0c76321c5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:43:29.376205 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.376170    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerID="391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2" exitCode=0
Sep 10 21:43:29.376351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.376214    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"
Sep 10 21:43:29.376351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.376250    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerDied","Data":"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"}
Sep 10 21:43:29.376351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.376304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2" event={"ID":"5b801ccc-499a-43d2-b92f-2e0c76321c5b","Type":"ContainerDied","Data":"efe4ca9a154d7608df1f3d1db23029fc045aed052705b612c8da124ee79164f2"}
Sep 10 21:43:29.376351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.376329    2579 scope.go:117] "RemoveContainer" containerID="391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"
Sep 10 21:43:29.388610 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.388592    2579 scope.go:117] "RemoveContainer" containerID="0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"
Sep 10 21:43:29.396611 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.396593    2579 scope.go:117] "RemoveContainer" containerID="391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"
Sep 10 21:43:29.396886 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:43:29.396869    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2\": container with ID starting with 391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2 not found: ID does not exist" containerID="391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"
Sep 10 21:43:29.396940 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.396894    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2"} err="failed to get container status \"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2\": rpc error: code = NotFound desc = could not find container \"391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2\": container with ID starting with 391ffd73e7d00a16e71b2286fc0dd1ee618772b54f3411f00b23d6eb06db86a2 not found: ID does not exist"
Sep 10 21:43:29.396940 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.396911    2579 scope.go:117] "RemoveContainer" containerID="0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"
Sep 10 21:43:29.397157 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:43:29.397140    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8\": container with ID starting with 0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8 not found: ID does not exist" containerID="0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"
Sep 10 21:43:29.397198 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.397163    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8"} err="failed to get container status \"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8\": rpc error: code = NotFound desc = could not find container \"0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8\": container with ID starting with 0bd63af051662694965aa794b2fdc9aeaf98d181815dd64b3043cf86c4e140b8 not found: ID does not exist"
Sep 10 21:43:29.401262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.401238    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:43:29.403582 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.403553    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-93f046-openshift-default-5d4758754d-p7wx2"]
Sep 10 21:43:29.426562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.426530    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.426562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.426560    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4pb2\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-kube-api-access-r4pb2\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.426562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.426570    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.426718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.426579    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.426718 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.426587    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b801ccc-499a-43d2-b92f-2e0c76321c5b-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:29.955974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:29.955941    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" path="/var/lib/kubelet/pods/5b801ccc-499a-43d2-b92f-2e0c76321c5b/volumes"
Sep 10 21:43:38.971000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:38.970971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:39.944795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:39.944754    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:39.984339 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:39.984304    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:43:39.984815 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:39.984759    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" podUID="477733d3-cf98-4fb5-b4f9-98d25084ed96" containerName="payload-processing" containerID="cri-o://a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff" gracePeriod=30
Sep 10 21:43:40.030154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.030091    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:40.134313 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.134287    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:40.217536 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.217508    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"]
Sep 10 21:43:40.220604 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.220578    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:40.221160 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.221140    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a867da-74qgv"]
Sep 10 21:43:40.229515 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.229495    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:43:40.292994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.292971    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:40.331936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.331904    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp4lm\" (UniqueName: \"kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm\") pod \"477733d3-cf98-4fb5-b4f9-98d25084ed96\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") "
Sep 10 21:43:40.332131 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.332016    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume\") pod \"477733d3-cf98-4fb5-b4f9-98d25084ed96\" (UID: \"477733d3-cf98-4fb5-b4f9-98d25084ed96\") "
Sep 10 21:43:40.332893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.332862    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "477733d3-cf98-4fb5-b4f9-98d25084ed96" (UID: "477733d3-cf98-4fb5-b4f9-98d25084ed96"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:40.337790 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.337721    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm" (OuterVolumeSpecName: "kube-api-access-zp4lm") pod "477733d3-cf98-4fb5-b4f9-98d25084ed96" (UID: "477733d3-cf98-4fb5-b4f9-98d25084ed96"). InnerVolumeSpecName "kube-api-access-zp4lm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:43:40.430920 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.430882    2579 generic.go:356] "Generic (PLEG): container finished" podID="477733d3-cf98-4fb5-b4f9-98d25084ed96" containerID="a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff" exitCode=0
Sep 10 21:43:40.431114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.430932    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"
Sep 10 21:43:40.431114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.430951    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" event={"ID":"477733d3-cf98-4fb5-b4f9-98d25084ed96","Type":"ContainerDied","Data":"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"}
Sep 10 21:43:40.431114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.431004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9" event={"ID":"477733d3-cf98-4fb5-b4f9-98d25084ed96","Type":"ContainerDied","Data":"52f177af479e408950224f8c2636fa408a07e6111c664a6c6da7bafe32df41fe"}
Sep 10 21:43:40.431114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.431035    2579 scope.go:117] "RemoveContainer" containerID="a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"
Sep 10 21:43:40.432703 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.432684    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/477733d3-cf98-4fb5-b4f9-98d25084ed96-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:40.432850 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.432705    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp4lm\" (UniqueName: \"kubernetes.io/projected/477733d3-cf98-4fb5-b4f9-98d25084ed96-kube-api-access-zp4lm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:40.440686 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.440669    2579 scope.go:117] "RemoveContainer" containerID="a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"
Sep 10 21:43:40.441000 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:43:40.440979    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff\": container with ID starting with a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff not found: ID does not exist" containerID="a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"
Sep 10 21:43:40.441065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.441009    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff"} err="failed to get container status \"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff\": rpc error: code = NotFound desc = could not find container \"a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff\": container with ID starting with a8df95f8523bcebd3484bec5f07c46797cafab56e668b214fd1c831c031e25ff not found: ID does not exist"
Sep 10 21:43:40.458185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.458100    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:43:40.463275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.463247    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a867da-5f459d5595-x59g9"]
Sep 10 21:43:40.602197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:40.602144    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:41.956165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:41.956130    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dcf88ec-164c-4ad4-be5e-d3e732e8b486" path="/var/lib/kubelet/pods/0dcf88ec-164c-4ad4-be5e-d3e732e8b486/volumes"
Sep 10 21:43:41.956533 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:41.956493    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477733d3-cf98-4fb5-b4f9-98d25084ed96" path="/var/lib/kubelet/pods/477733d3-cf98-4fb5-b4f9-98d25084ed96/volumes"
Sep 10 21:43:46.591665 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:46.591631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:46.692389 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:46.692354    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:46.931908 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:46.931822    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:43:46.932080 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:46.932045    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" containerID="cri-o://52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4" gracePeriod=30
Sep 10 21:43:46.981124 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:46.981095    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:47.232701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.232621    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"]
Sep 10 21:43:47.232996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.232982    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:43:47.233048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.232998    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:43:47.233048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233009    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dcf88ec-164c-4ad4-be5e-d3e732e8b486" containerName="revoke-keys"
Sep 10 21:43:47.233048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233015    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dcf88ec-164c-4ad4-be5e-d3e732e8b486" containerName="revoke-keys"
Sep 10 21:43:47.233048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233029    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477733d3-cf98-4fb5-b4f9-98d25084ed96" containerName="payload-processing"
Sep 10 21:43:47.233048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233035    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="477733d3-cf98-4fb5-b4f9-98d25084ed96" containerName="payload-processing"
Sep 10 21:43:47.233197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233097    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:43:47.233197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233105    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dcf88ec-164c-4ad4-be5e-d3e732e8b486" containerName="revoke-keys"
Sep 10 21:43:47.233197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233111    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="477733d3-cf98-4fb5-b4f9-98d25084ed96" containerName="payload-processing"
Sep 10 21:43:47.233197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.233116    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:43:47.237427 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.237409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:47.239462 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.239442    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:43:47.239555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.239536    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:43:47.244092 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.244067    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"]
Sep 10 21:43:47.346053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.346022    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:47.394651 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.394621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z\" (UID: \"e744b262-01da-4420-a5ff-b7389fb5743e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:47.495704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.495608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z\" (UID: \"e744b262-01da-4420-a5ff-b7389fb5743e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:47.496295 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.496268    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z\" (UID: \"e744b262-01da-4420-a5ff-b7389fb5743e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:47.549602 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.549570    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:47.821637 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:47.821612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"]
Sep 10 21:43:47.823954 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:43:47.823931    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode744b262_01da_4420_a5ff_b7389fb5743e.slice/crio-874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704 WatchSource:0}: Error finding container 874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704: Status 404 returned error can't find the container with id 874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704
Sep 10 21:43:48.471263 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:48.471176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z" event={"ID":"e744b262-01da-4420-a5ff-b7389fb5743e","Type":"ContainerStarted","Data":"719972ac1f5c3ec1b060a1b7d19a6b6e37e2b0d6c8e8e13f81eec2465d818ac8"}
Sep 10 21:43:48.471263 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:48.471260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z" event={"ID":"e744b262-01da-4420-a5ff-b7389fb5743e","Type":"ContainerStarted","Data":"874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704"}
Sep 10 21:43:48.487727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:48.487670    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z" podStartSLOduration=1.487655696 podStartE2EDuration="1.487655696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:43:48.486148962 +0000 UTC m=+1464.980972501" watchObservedRunningTime="2026-09-10 21:43:48.487655696 +0000 UTC m=+1464.982479234"
Sep 10 21:43:48.862056 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:48.862019    2579 patch_prober.go:28] interesting pod/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:43:48.862426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:48.862093    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:43:49.477355 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:49.477324    2579 generic.go:356] "Generic (PLEG): container finished" podID="e744b262-01da-4420-a5ff-b7389fb5743e" containerID="719972ac1f5c3ec1b060a1b7d19a6b6e37e2b0d6c8e8e13f81eec2465d818ac8" exitCode=0
Sep 10 21:43:49.477543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:49.477369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z" event={"ID":"e744b262-01da-4420-a5ff-b7389fb5743e","Type":"ContainerDied","Data":"719972ac1f5c3ec1b060a1b7d19a6b6e37e2b0d6c8e8e13f81eec2465d818ac8"}
Sep 10 21:43:50.618469 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:50.618443    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:50.729707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:50.729667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca\") pod \"e744b262-01da-4420-a5ff-b7389fb5743e\" (UID: \"e744b262-01da-4420-a5ff-b7389fb5743e\") "
Sep 10 21:43:50.730076 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:50.730053    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e744b262-01da-4420-a5ff-b7389fb5743e" (UID: "e744b262-01da-4420-a5ff-b7389fb5743e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:50.831047 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:50.831016    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e744b262-01da-4420-a5ff-b7389fb5743e-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:51.487855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:51.487823    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"
Sep 10 21:43:51.488035 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:51.487857    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z" event={"ID":"e744b262-01da-4420-a5ff-b7389fb5743e","Type":"ContainerDied","Data":"874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704"}
Sep 10 21:43:51.488035 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:51.487904    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="874548afb0e6b6d241afc49fa2af922ded6bde03d80fafe8956ee346629fa704"
Sep 10 21:43:52.199681 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.199596    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:43:52.242680 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.242892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.242892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242732    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.242892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.242892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242827    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.242892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242859    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glr6h\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.243157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242900    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.243157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.242997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:52.243157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.243110    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:52.243309 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.243202    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data" (OuterVolumeSpecName: "istio-data") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:52.243309 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.243270    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.243309 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.243291    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.245348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.245321    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:52.245527 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.245434    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:43:52.245860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.245825    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h" (OuterVolumeSpecName: "kube-api-access-glr6h") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "kube-api-access-glr6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:43:52.245972 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.245881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token" (OuterVolumeSpecName: "istio-token") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:43:52.343621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343568    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.343621 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343626    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert\") pod \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\" (UID: \"adc9f573-37b1-4b3a-ac98-ee731f97ab73\") "
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343841    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343851    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343859    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343869    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glr6h\" (UniqueName: \"kubernetes.io/projected/adc9f573-37b1-4b3a-ac98-ee731f97ab73-kube-api-access-glr6h\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343877    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.343938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.343882    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:43:52.344205 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.344077    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "adc9f573-37b1-4b3a-ac98-ee731f97ab73" (UID: "adc9f573-37b1-4b3a-ac98-ee731f97ab73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:43:52.445179 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.445137    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adc9f573-37b1-4b3a-ac98-ee731f97ab73-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.445179 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.445167    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adc9f573-37b1-4b3a-ac98-ee731f97ab73-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:43:52.495351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.495253    2579 generic.go:356] "Generic (PLEG): container finished" podID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerID="52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4" exitCode=0
Sep 10 21:43:52.495351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.495310    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"
Sep 10 21:43:52.495351 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.495332    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerDied","Data":"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"}
Sep 10 21:43:52.495624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.495393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2" event={"ID":"adc9f573-37b1-4b3a-ac98-ee731f97ab73","Type":"ContainerDied","Data":"7aefc997cc48322a13895116ec0692936d37146ca2f1feba365d1fb1821dc3d3"}
Sep 10 21:43:52.495624 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.495424    2579 scope.go:117] "RemoveContainer" containerID="52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"
Sep 10 21:43:52.507175 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.507156    2579 scope.go:117] "RemoveContainer" containerID="9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"
Sep 10 21:43:52.516195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.516172    2579 scope.go:117] "RemoveContainer" containerID="52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"
Sep 10 21:43:52.516541 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:43:52.516520    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4\": container with ID starting with 52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4 not found: ID does not exist" containerID="52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"
Sep 10 21:43:52.516603 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.516552    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4"} err="failed to get container status \"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4\": rpc error: code = NotFound desc = could not find container \"52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4\": container with ID starting with 52912370a10848ec88a15ef65eed709903abb8b3c47c77fbee3a38195c3818c4 not found: ID does not exist"
Sep 10 21:43:52.516603 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.516571    2579 scope.go:117] "RemoveContainer" containerID="9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"
Sep 10 21:43:52.516874 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:43:52.516853    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82\": container with ID starting with 9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82 not found: ID does not exist" containerID="9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"
Sep 10 21:43:52.516939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.516873    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:43:52.516939 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.516882    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82"} err="failed to get container status \"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82\": rpc error: code = NotFound desc = could not find container \"9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82\": container with ID starting with 9700e5284d312f84b6ecfc202ea405e6a4d6e7ce4d512b6866281ccb0bfc8f82 not found: ID does not exist"
Sep 10 21:43:52.520364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:52.520342    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a867da-openshift-default-56c8b9fcc6-w8cl2"]
Sep 10 21:43:53.956000 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:53.955963    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" path="/var/lib/kubelet/pods/adc9f573-37b1-4b3a-ac98-ee731f97ab73/volumes"
Sep 10 21:43:57.775895 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:57.775865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:57.878194 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:57.878162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:58.021293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:58.021265    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:58.119443 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:58.119368    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"]
Sep 10 21:43:58.128048 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:58.128012    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-4b45c2-5fh5z"]
Sep 10 21:43:58.148931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:58.148909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:58.432972 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:58.432876    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:43:59.956004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:43:59.955971    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e744b262-01da-4420-a5ff-b7389fb5743e" path="/var/lib/kubelet/pods/e744b262-01da-4420-a5ff-b7389fb5743e/volumes"
Sep 10 21:44:04.522526 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:04.522471    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:04.623053 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:04.623028    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:04.734436 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:04.734409    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:44:04.734646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:04.734618    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy" containerID="cri-o://ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be" gracePeriod=30
Sep 10 21:44:04.758945 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:04.758920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:05.088200 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088166    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"]
Sep 10 21:44:05.088559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088545    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:05.088606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088562    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:05.088606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088596    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:44:05.088606 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088602    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b801ccc-499a-43d2-b92f-2e0c76321c5b" containerName="istio-proxy"
Sep 10 21:44:05.088701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088613    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e744b262-01da-4420-a5ff-b7389fb5743e" containerName="revoke-keys"
Sep 10 21:44:05.088701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088618    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e744b262-01da-4420-a5ff-b7389fb5743e" containerName="revoke-keys"
Sep 10 21:44:05.088701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088673    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:05.088701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.088683    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e744b262-01da-4420-a5ff-b7389fb5743e" containerName="revoke-keys"
Sep 10 21:44:05.091950 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.091935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:05.094223 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.094200    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:44:05.094337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.094259    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:44:05.100115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.100092    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"]
Sep 10 21:44:05.156148 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.156117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp\" (UID: \"a3d1fcbc-f431-4a67-982c-486c6c784360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:05.257197 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.257161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp\" (UID: \"a3d1fcbc-f431-4a67-982c-486c6c784360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:05.257788 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.257756    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp\" (UID: \"a3d1fcbc-f431-4a67-982c-486c6c784360\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:05.347154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.347034    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:05.403038 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.402992    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:05.687538 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:05.687485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"]
Sep 10 21:44:05.689841 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:44:05.689812    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d1fcbc_f431_4a67_982c_486c6c784360.slice/crio-a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63 WatchSource:0}: Error finding container a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63: Status 404 returned error can't find the container with id a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63
Sep 10 21:44:06.567749 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:06.567714    2579 generic.go:356] "Generic (PLEG): container finished" podID="a3d1fcbc-f431-4a67-982c-486c6c784360" containerID="f8c591c327ff2488611d506c720ee93c6a708efd5d4732394ca8fc57761a9a2b" exitCode=0
Sep 10 21:44:06.567949 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:06.567759    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp" event={"ID":"a3d1fcbc-f431-4a67-982c-486c6c784360","Type":"ContainerDied","Data":"f8c591c327ff2488611d506c720ee93c6a708efd5d4732394ca8fc57761a9a2b"}
Sep 10 21:44:06.567949 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:06.567817    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp" event={"ID":"a3d1fcbc-f431-4a67-982c-486c6c784360","Type":"ContainerStarted","Data":"a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63"}
Sep 10 21:44:07.701744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:07.701723    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:07.778453 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:07.778426    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca\") pod \"a3d1fcbc-f431-4a67-982c-486c6c784360\" (UID: \"a3d1fcbc-f431-4a67-982c-486c6c784360\") "
Sep 10 21:44:07.778756 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:07.778734    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3d1fcbc-f431-4a67-982c-486c6c784360" (UID: "a3d1fcbc-f431-4a67-982c-486c6c784360"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:07.879592 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:07.879531    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3d1fcbc-f431-4a67-982c-486c6c784360-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:08.578950 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:08.578915    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp" event={"ID":"a3d1fcbc-f431-4a67-982c-486c6c784360","Type":"ContainerDied","Data":"a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63"}
Sep 10 21:44:08.578950 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:08.578944    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"
Sep 10 21:44:08.579133 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:08.578965    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a38d71548a7fea890d089df99f51848fa58b33b79a2ecda3615a234d766e1b63"
Sep 10 21:44:09.992483 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:09.992461    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:44:10.099250 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099219    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099250 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099275    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099362    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099419    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099442    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099723 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4xr2\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099723 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099497    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket\") pod \"d45ec884-1a31-4443-a627-b729d3752af0\" (UID: \"d45ec884-1a31-4443-a627-b729d3752af0\") "
Sep 10 21:44:10.099723 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099493    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:10.099936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099729    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data" (OuterVolumeSpecName: "istio-data") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:10.099936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099743    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:10.099936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099833    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:10.099936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099918    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.099936 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099937    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.100190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099950    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.100190 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.099970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:10.101753 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.101721    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:44:10.101917 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.101764    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:44:10.101980 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.101960    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token" (OuterVolumeSpecName: "istio-token") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:10.102076 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.102054    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2" (OuterVolumeSpecName: "kube-api-access-d4xr2") pod "d45ec884-1a31-4443-a627-b729d3752af0" (UID: "d45ec884-1a31-4443-a627-b729d3752af0"). InnerVolumeSpecName "kube-api-access-d4xr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:10.201311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201279    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d45ec884-1a31-4443-a627-b729d3752af0-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.201311 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201306    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.201557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201320    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d45ec884-1a31-4443-a627-b729d3752af0-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.201557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201331    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.201557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201341    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4xr2\" (UniqueName: \"kubernetes.io/projected/d45ec884-1a31-4443-a627-b729d3752af0-kube-api-access-d4xr2\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.201557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.201353    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d45ec884-1a31-4443-a627-b729d3752af0-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:10.589912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.589882    2579 generic.go:356] "Generic (PLEG): container finished" podID="d45ec884-1a31-4443-a627-b729d3752af0" containerID="ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be" exitCode=0
Sep 10 21:44:10.590085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.589928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"
Sep 10 21:44:10.590085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.589962    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerDied","Data":"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"}
Sep 10 21:44:10.590085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.590002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk" event={"ID":"d45ec884-1a31-4443-a627-b729d3752af0","Type":"ContainerDied","Data":"997da1947f2b2eb8eff1cc6e9178f3655630cb3d5a30599f9fef9c6a796f436b"}
Sep 10 21:44:10.590085 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.590025    2579 scope.go:117] "RemoveContainer" containerID="ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"
Sep 10 21:44:10.599195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.599181    2579 scope.go:117] "RemoveContainer" containerID="55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"
Sep 10 21:44:10.606791 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.606757    2579 scope.go:117] "RemoveContainer" containerID="ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"
Sep 10 21:44:10.607077 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:44:10.607055    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be\": container with ID starting with ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be not found: ID does not exist" containerID="ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"
Sep 10 21:44:10.607175 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.607083    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be"} err="failed to get container status \"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be\": rpc error: code = NotFound desc = could not find container \"ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be\": container with ID starting with ac01f674618f254ab95f734b50ba0a8220bc670fdbb28bb6053a8e1a5ae722be not found: ID does not exist"
Sep 10 21:44:10.607175 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.607099    2579 scope.go:117] "RemoveContainer" containerID="55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"
Sep 10 21:44:10.607481 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:44:10.607439    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a\": container with ID starting with 55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a not found: ID does not exist" containerID="55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"
Sep 10 21:44:10.607588 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.607471    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a"} err="failed to get container status \"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a\": rpc error: code = NotFound desc = could not find container \"55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a\": container with ID starting with 55d0350ec0c1c4ea6c15a9513a0a43396b9192023b97f8c1d154630e3eda433a not found: ID does not exist"
Sep 10 21:44:10.609693 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.609673    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:44:10.614155 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:10.614134    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-4b45c2-openshift-default-9c4c9b4db-7n5bk"]
Sep 10 21:44:11.955793 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:11.955750    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d45ec884-1a31-4443-a627-b729d3752af0" path="/var/lib/kubelet/pods/d45ec884-1a31-4443-a627-b729d3752af0/volumes"
Sep 10 21:44:15.654451 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.654408    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:15.676825 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.676797    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:44:15.677062 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.677037    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" podUID="95429d86-01cc-410b-a4c1-cd857026c15b" containerName="payload-processing" containerID="cri-o://b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9" gracePeriod=30
Sep 10 21:44:15.684435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.684051    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:44:15.684435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.684280    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" podUID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" containerName="payload-pre-processing" containerID="cri-o://4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3" gracePeriod=30
Sep 10 21:44:15.724744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.724717    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:15.824459 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.824431    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:15.851391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.851360    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"]
Sep 10 21:44:15.854187 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.854157    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-b4c9bb-9qrhp"]
Sep 10 21:44:15.896742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.896710    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:15.955958 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.955933    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3d1fcbc-f431-4a67-982c-486c6c784360" path="/var/lib/kubelet/pods/a3d1fcbc-f431-4a67-982c-486c6c784360/volumes"
Sep 10 21:44:15.956457 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.956439    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:44:15.959859 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:15.959841    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:44:16.055891 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.055854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume\") pod \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") "
Sep 10 21:44:16.055891 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.055894    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9llqs\" (UniqueName: \"kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs\") pod \"95429d86-01cc-410b-a4c1-cd857026c15b\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") "
Sep 10 21:44:16.056157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.055930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume\") pod \"95429d86-01cc-410b-a4c1-cd857026c15b\" (UID: \"95429d86-01cc-410b-a4c1-cd857026c15b\") "
Sep 10 21:44:16.056157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.055970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k85b\" (UniqueName: \"kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b\") pod \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\" (UID: \"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb\") "
Sep 10 21:44:16.056327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.056299    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" (UID: "0a74c279-1c1c-4cc9-82c7-6fc336fb62eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:16.056327 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.056322    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95429d86-01cc-410b-a4c1-cd857026c15b" (UID: "95429d86-01cc-410b-a4c1-cd857026c15b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:44:16.058154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.058134    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b" (OuterVolumeSpecName: "kube-api-access-9k85b") pod "0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" (UID: "0a74c279-1c1c-4cc9-82c7-6fc336fb62eb"). InnerVolumeSpecName "kube-api-access-9k85b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:16.058252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.058228    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs" (OuterVolumeSpecName: "kube-api-access-9llqs") pod "95429d86-01cc-410b-a4c1-cd857026c15b" (UID: "95429d86-01cc-410b-a4c1-cd857026c15b"). InnerVolumeSpecName "kube-api-access-9llqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:44:16.157264 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.157241    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:16.157264 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.157264    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9llqs\" (UniqueName: \"kubernetes.io/projected/95429d86-01cc-410b-a4c1-cd857026c15b-kube-api-access-9llqs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:16.157411 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.157274    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95429d86-01cc-410b-a4c1-cd857026c15b-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:16.157411 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.157282    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k85b\" (UniqueName: \"kubernetes.io/projected/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb-kube-api-access-9k85b\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:44:16.215207 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.215147    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:16.622482 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.622449    2579 generic.go:356] "Generic (PLEG): container finished" podID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" containerID="4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3" exitCode=0
Sep 10 21:44:16.622664 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.622494    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"
Sep 10 21:44:16.622664 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.622530    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" event={"ID":"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb","Type":"ContainerDied","Data":"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"}
Sep 10 21:44:16.622664 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.622596    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk" event={"ID":"0a74c279-1c1c-4cc9-82c7-6fc336fb62eb","Type":"ContainerDied","Data":"3c0251a3523a7cf99d5df053dce40584cec9c09a2cc1a1f862ec6b6644c5e39a"}
Sep 10 21:44:16.622664 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.622633    2579 scope.go:117] "RemoveContainer" containerID="4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"
Sep 10 21:44:16.623985 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.623964    2579 generic.go:356] "Generic (PLEG): container finished" podID="95429d86-01cc-410b-a4c1-cd857026c15b" containerID="b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9" exitCode=0
Sep 10 21:44:16.624108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.624001    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"
Sep 10 21:44:16.624108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.624045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" event={"ID":"95429d86-01cc-410b-a4c1-cd857026c15b","Type":"ContainerDied","Data":"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"}
Sep 10 21:44:16.624108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.624092    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr" event={"ID":"95429d86-01cc-410b-a4c1-cd857026c15b","Type":"ContainerDied","Data":"5421ea37540272c52b6c2bc77c439142aefda5f0d60e2b445ceec16544924f4a"}
Sep 10 21:44:16.632279 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.632262    2579 scope.go:117] "RemoveContainer" containerID="4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"
Sep 10 21:44:16.632603 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:44:16.632581    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3\": container with ID starting with 4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3 not found: ID does not exist" containerID="4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"
Sep 10 21:44:16.632697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.632607    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3"} err="failed to get container status \"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3\": rpc error: code = NotFound desc = could not find container \"4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3\": container with ID starting with 4cf4e1647c3f5662e5239b34f51c61ed54f9836ea63fabb16be32a3f27683fe3 not found: ID does not exist"
Sep 10 21:44:16.632697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.632621    2579 scope.go:117] "RemoveContainer" containerID="b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"
Sep 10 21:44:16.640657 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.640641    2579 scope.go:117] "RemoveContainer" containerID="b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"
Sep 10 21:44:16.640949 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:44:16.640931    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9\": container with ID starting with b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9 not found: ID does not exist" containerID="b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"
Sep 10 21:44:16.641027 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.640953    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9"} err="failed to get container status \"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9\": rpc error: code = NotFound desc = could not find container \"b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9\": container with ID starting with b168237198ee7a26cea4069523fe110b1684fe882110e9af798204a18d1470a9 not found: ID does not exist"
Sep 10 21:44:16.646968 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.646949    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:44:16.651165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.651146    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b4c9bb-6cf96d76c8-29xxk"]
Sep 10 21:44:16.660762 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.660733    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:44:16.667112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:16.667094    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-b4c9bb-5749fb5bc5-cwdtr"]
Sep 10 21:44:17.436589 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:17.436535    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:17.954698 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:17.954668    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" path="/var/lib/kubelet/pods/0a74c279-1c1c-4cc9-82c7-6fc336fb62eb/volumes"
Sep 10 21:44:17.955077 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:17.955032    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95429d86-01cc-410b-a4c1-cd857026c15b" path="/var/lib/kubelet/pods/95429d86-01cc-410b-a4c1-cd857026c15b/volumes"
Sep 10 21:44:22.522894 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:22.522851    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:22.602792 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:22.602728    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:22.783159 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:22.783066    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:27.182848 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:27.182808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:28.119564 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.119524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:28.556281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.556249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:28.848741 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.848669    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:44:28.849027 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849013    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849029    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849039    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3d1fcbc-f431-4a67-982c-486c6c784360" containerName="revoke-keys"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849044    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3d1fcbc-f431-4a67-982c-486c6c784360" containerName="revoke-keys"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849052    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" containerName="payload-pre-processing"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849058    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" containerName="payload-pre-processing"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849063    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95429d86-01cc-410b-a4c1-cd857026c15b" containerName="payload-processing"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849068    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="95429d86-01cc-410b-a4c1-cd857026c15b" containerName="payload-processing"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849073    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:28.849082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849077    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849087    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849092    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849152    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a74c279-1c1c-4cc9-82c7-6fc336fb62eb" containerName="payload-pre-processing"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849161    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849168    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="95429d86-01cc-410b-a4c1-cd857026c15b" containerName="payload-processing"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849173    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3d1fcbc-f431-4a67-982c-486c6c784360" containerName="revoke-keys"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849179    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d45ec884-1a31-4443-a627-b729d3752af0" containerName="istio-proxy"
Sep 10 21:44:28.849343 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.849185    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="adc9f573-37b1-4b3a-ac98-ee731f97ab73" containerName="istio-proxy"
Sep 10 21:44:28.853626 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.853593    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.855993 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.855971    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-8291ef-openshift-default-dockercfg-zm9c8\""
Sep 10 21:44:28.868698 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.868675    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:44:28.883018 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.882996    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:28.977417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977417 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977422    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977527    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977620 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:28.977720 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:28.977709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j62pz\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.018664 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.018632    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078608    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j62pz\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078768    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.078963    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.079001    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.079032    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.080092    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.080097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.080336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.082905    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.083148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.083270 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.083202    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.084107 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.083376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.088418 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.088393    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j62pz\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.091299 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.091277    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token\") pod \"e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") " pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.165525 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.165440    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:29.440517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.440495    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:44:29.442559 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:44:29.442525    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9ca713_897d_4558_b42a_1087e06de374.slice/crio-f61cbe6018643c6cf8e6c8398d6dd6ce20955f36447a44d997b8d9c0b51c8704 WatchSource:0}: Error finding container f61cbe6018643c6cf8e6c8398d6dd6ce20955f36447a44d997b8d9c0b51c8704: Status 404 returned error can't find the container with id f61cbe6018643c6cf8e6c8398d6dd6ce20955f36447a44d997b8d9c0b51c8704
Sep 10 21:44:29.444730 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.444696    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:29.444828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.444762    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:29.444828 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.444813    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:29.689336 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.689301    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerStarted","Data":"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"}
Sep 10 21:44:29.689812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.689360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerStarted","Data":"f61cbe6018643c6cf8e6c8398d6dd6ce20955f36447a44d997b8d9c0b51c8704"}
Sep 10 21:44:29.712855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:29.712701    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podStartSLOduration=1.7126815720000002 podStartE2EDuration="1.712681572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:44:29.710882191 +0000 UTC m=+1506.205705732" watchObservedRunningTime="2026-09-10 21:44:29.712681572 +0000 UTC m=+1506.207505112"
Sep 10 21:44:30.165870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:30.165831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:30.171395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:30.171368    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:30.694175 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:30.694139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:30.695125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:30.695110    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:31.625651 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:31.625617    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:31.723272 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:31.723242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:32.940745 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:32.940718    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:33.020134 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:33.019243    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:33.159928 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:33.159897    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:33.535186 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:33.535155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:33.631994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:33.631968    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:46.734987 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:46.734899    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 21:44:46.735397 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:46.734980    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 21:44:47.782142 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.782108    2579 generic.go:356] "Generic (PLEG): container finished" podID="af9ca713-897d-4558-b42a-1087e06de374" containerID="01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e" exitCode=0
Sep 10 21:44:47.782547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.782183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerDied","Data":"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"}
Sep 10 21:44:47.782547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.782529    2579 scope.go:117] "RemoveContainer" containerID="01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"
Sep 10 21:44:47.783603 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.783570    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:47.783721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.783642    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:47.783721 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:47.783673    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:44:48.788551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:48.788519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerStarted","Data":"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"}
Sep 10 21:44:49.166694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:49.166611    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:49.166865 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:49.166828    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 10 21:44:49.166911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:49.166874    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 10 21:44:50.166548 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:50.166508    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 10 21:44:50.167104 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:50.166591    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 10 21:44:51.166269 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:51.166230    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 10 21:44:51.166450 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:51.166288    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 10 21:44:52.166498 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:52.166466    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 10 21:44:52.166930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:52.166522    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 10 21:44:53.166812 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:53.166757    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 10 21:44:53.167201 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:53.166848    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 10 21:44:54.169903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:54.169874    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:54.170306 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:54.170248    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:54.170876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:54.170857    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:44:56.607573 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:56.607536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:57.273507 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:57.273477    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:57.638602 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:57.638527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:44:57.767024 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:44:57.766983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:00.137745 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.137706    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"]
Sep 10 21:45:00.141108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.141082    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"
Sep 10 21:45:00.141275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.141205    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"]
Sep 10 21:45:00.143652 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.143621    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:45:00.144722 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.144698    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"
Sep 10 21:45:00.145595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.145571    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"]
Sep 10 21:45:00.149103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.149064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"]
Sep 10 21:45:00.149245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.149162    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"
Sep 10 21:45:00.154886 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.154861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"
Sep 10 21:45:00.156326 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.156271    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"]
Sep 10 21:45:00.159969 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.159943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"]
Sep 10 21:45:00.168123 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.168103    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"
Sep 10 21:45:00.201287 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.201264    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"
Sep 10 21:45:00.329700 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.329651    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:00.430250 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.430183    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:00.461922 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.453695    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"]
Sep 10 21:45:00.470294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.470256    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"]
Sep 10 21:45:00.471364 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:45:00.471339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9184245d_af57_40d4_825f_9bb4d0536546.slice/crio-50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8 WatchSource:0}: Error finding container 50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8: Status 404 returned error can't find the container with id 50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8
Sep 10 21:45:00.492466 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.492447    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"]
Sep 10 21:45:00.497842 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:45:00.497814    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded63665f_ffe0_4afa_a625_ab0613099c28.slice/crio-597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c WatchSource:0}: Error finding container 597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c: Status 404 returned error can't find the container with id 597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c
Sep 10 21:45:00.846491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.846463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf" event={"ID":"9184245d-af57-40d4-825f-9bb4d0536546","Type":"ContainerStarted","Data":"30f4f985a1d0a89acc15afb1df19f373e59325b52bd221c14584031ef886a999"}
Sep 10 21:45:00.846616 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.846518    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf" event={"ID":"9184245d-af57-40d4-825f-9bb4d0536546","Type":"ContainerStarted","Data":"50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8"}
Sep 10 21:45:00.848365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.848336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw" event={"ID":"ed63665f-ffe0-4afa-a625-ab0613099c28","Type":"ContainerStarted","Data":"597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c"}
Sep 10 21:45:00.850112 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.850085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz" event={"ID":"32a7c823-a790-4545-a0c1-c6bdd6360455","Type":"ContainerStarted","Data":"f271b96c2b26f6f19d1deb79101204338eb85b8029f5d8f7c1788331032fa599"}
Sep 10 21:45:00.850229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.850136    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz" event={"ID":"32a7c823-a790-4545-a0c1-c6bdd6360455","Type":"ContainerStarted","Data":"e63a0c89a638777a49fe2cb673148144087738d1cc55eb4c98e69ca5a0b0ad20"}
Sep 10 21:45:00.864797 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.864737    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf" podStartSLOduration=0.864723994 podStartE2EDuration="864.723994ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:00.863220989 +0000 UTC m=+1537.358044528" watchObservedRunningTime="2026-09-10 21:45:00.864723994 +0000 UTC m=+1537.359547532"
Sep 10 21:45:00.879394 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:00.879356    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz" podStartSLOduration=0.879340802 podStartE2EDuration="879.340802ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:00.878517787 +0000 UTC m=+1537.373341325" watchObservedRunningTime="2026-09-10 21:45:00.879340802 +0000 UTC m=+1537.374164359"
Sep 10 21:45:01.513543 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.513510    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:01.857262 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.857230    2579 generic.go:356] "Generic (PLEG): container finished" podID="ed63665f-ffe0-4afa-a625-ab0613099c28" containerID="3e35f0bc9b199865c3d100e45c3ab99f7a1b75b25c4cc98a14c7f487d9da2a0c" exitCode=0
Sep 10 21:45:01.857446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.857295    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw" event={"ID":"ed63665f-ffe0-4afa-a625-ab0613099c28","Type":"ContainerDied","Data":"3e35f0bc9b199865c3d100e45c3ab99f7a1b75b25c4cc98a14c7f487d9da2a0c"}
Sep 10 21:45:01.858986 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.858961    2579 generic.go:356] "Generic (PLEG): container finished" podID="32a7c823-a790-4545-a0c1-c6bdd6360455" containerID="f271b96c2b26f6f19d1deb79101204338eb85b8029f5d8f7c1788331032fa599" exitCode=0
Sep 10 21:45:01.859122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.859034    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz" event={"ID":"32a7c823-a790-4545-a0c1-c6bdd6360455","Type":"ContainerDied","Data":"f271b96c2b26f6f19d1deb79101204338eb85b8029f5d8f7c1788331032fa599"}
Sep 10 21:45:01.860735 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.860714    2579 generic.go:356] "Generic (PLEG): container finished" podID="9184245d-af57-40d4-825f-9bb4d0536546" containerID="30f4f985a1d0a89acc15afb1df19f373e59325b52bd221c14584031ef886a999" exitCode=0
Sep 10 21:45:01.860877 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:01.860854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf" event={"ID":"9184245d-af57-40d4-825f-9bb4d0536546","Type":"ContainerDied","Data":"30f4f985a1d0a89acc15afb1df19f373e59325b52bd221c14584031ef886a999"}
Sep 10 21:45:02.093242 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.093212    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:45:02.096407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.096385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.100754 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.100727    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2fbf53-dockercfg-4rs9d\""
Sep 10 21:45:02.100900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.100847    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2fbf53\""
Sep 10 21:45:02.110310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.110244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:45:02.184463 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.184429    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9fc7\" (UniqueName: \"kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.184646 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.184502    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.285956 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.285920    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9fc7\" (UniqueName: \"kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.286135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.286004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.286679 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.286659    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.294003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.293975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9fc7\" (UniqueName: \"kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7\") pod \"payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.408898 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.408813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.676940 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.676917    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:45:02.679542 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:45:02.679516    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod001e129a_f2f8_47a4_89a8_7f5fc6c8aa1e.slice/crio-7060121292be359767f63b1e8fb248bfd1e925e93690047e6dd1db7f1a53c99a WatchSource:0}: Error finding container 7060121292be359767f63b1e8fb248bfd1e925e93690047e6dd1db7f1a53c99a: Status 404 returned error can't find the container with id 7060121292be359767f63b1e8fb248bfd1e925e93690047e6dd1db7f1a53c99a
Sep 10 21:45:02.866861 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.866832    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" event={"ID":"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e","Type":"ContainerStarted","Data":"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"}
Sep 10 21:45:02.867003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.866889    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" event={"ID":"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e","Type":"ContainerStarted","Data":"7060121292be359767f63b1e8fb248bfd1e925e93690047e6dd1db7f1a53c99a"}
Sep 10 21:45:02.867003 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.866951    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:02.902229 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:02.902015    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" podStartSLOduration=0.901993375 podStartE2EDuration="901.993375ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:45:02.8999866 +0000 UTC m=+1539.394810140" watchObservedRunningTime="2026-09-10 21:45:02.901993375 +0000 UTC m=+1539.396816913"
Sep 10 21:45:03.008855 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.008829    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"
Sep 10 21:45:03.012535 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.012512    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"
Sep 10 21:45:03.034402 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.033734    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:03.057416 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.057393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"
Sep 10 21:45:03.872522 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.872495    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"
Sep 10 21:45:03.872955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.872498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw" event={"ID":"ed63665f-ffe0-4afa-a625-ab0613099c28","Type":"ContainerDied","Data":"597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c"}
Sep 10 21:45:03.872955 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.872618    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="597193c2d80e99e060933be481d6f1165394552f53f3d7f5bb7eb90236010d4c"
Sep 10 21:45:03.873930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.873907    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz"
Sep 10 21:45:03.874065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.873912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29817945-dx5fz" event={"ID":"32a7c823-a790-4545-a0c1-c6bdd6360455","Type":"ContainerDied","Data":"e63a0c89a638777a49fe2cb673148144087738d1cc55eb4c98e69ca5a0b0ad20"}
Sep 10 21:45:03.874065 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.874048    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e63a0c89a638777a49fe2cb673148144087738d1cc55eb4c98e69ca5a0b0ad20"
Sep 10 21:45:03.883407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.883371    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf" event={"ID":"9184245d-af57-40d4-825f-9bb4d0536546","Type":"ContainerDied","Data":"50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8"}
Sep 10 21:45:03.883552 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.883447    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"
Sep 10 21:45:03.883552 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:03.883464    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50111e94e23b9f8c4f389175d095228b92be70de4cbebac3aec48307294835b8"
Sep 10 21:45:13.885399 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:13.885371    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:45:25.349199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:25.349162    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:45:33.303866 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:45:33.303830    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:11.116370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:11.116338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:11.218760 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:11.218732    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:11.749461 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:11.749433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:12.034914 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.034886    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:46:12.035228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035213    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9184245d-af57-40d4-825f-9bb4d0536546" containerName="cleanup"
Sep 10 21:46:12.035228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035229    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9184245d-af57-40d4-825f-9bb4d0536546" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035245    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed63665f-ffe0-4afa-a625-ab0613099c28" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035251    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed63665f-ffe0-4afa-a625-ab0613099c28" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035260    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a7c823-a790-4545-a0c1-c6bdd6360455" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035265    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a7c823-a790-4545-a0c1-c6bdd6360455" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035325    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9184245d-af57-40d4-825f-9bb4d0536546" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035333    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed63665f-ffe0-4afa-a625-ab0613099c28" containerName="cleanup"
Sep 10 21:46:12.035353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.035341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a7c823-a790-4545-a0c1-c6bdd6360455" containerName="cleanup"
Sep 10 21:46:12.038519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.038500    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.040648 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.040623    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-bf96d8-openshift-default-dockercfg-4s2s8\""
Sep 10 21:46:12.050977 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.050956    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:46:12.076650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.076622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:12.182806 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.182744    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.182803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.182845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.182862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlj5k\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.182994    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.183033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.183059    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.183096    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.183277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.183135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.226696 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.226664    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:12.284149 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284108    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284154    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284181    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284294 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlj5k\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284340    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284411    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.284967 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.284919    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.285181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.285157    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.285695 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.285669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.285935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.285914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.286241 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.286219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.289569 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.289510    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.289569 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.289515    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.294269 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.294245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlj5k\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.294382 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.294356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token\") pod \"e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") " pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.349395 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.349371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:12.621764 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.621733    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:46:12.624848 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:46:12.624815    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f77a28_238d_41de_8ccd_7fb8ef98468b.slice/crio-2af256058e8eaa88b8aa86f53cdf53cd24a22cc14a28418a0e4abe77c57a4a24 WatchSource:0}: Error finding container 2af256058e8eaa88b8aa86f53cdf53cd24a22cc14a28418a0e4abe77c57a4a24: Status 404 returned error can't find the container with id 2af256058e8eaa88b8aa86f53cdf53cd24a22cc14a28418a0e4abe77c57a4a24
Sep 10 21:46:12.628296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.628263    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:12.628409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.628344    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:12.628409 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:12.628386    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:13.218715 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:13.218670    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerStarted","Data":"52348fea5c927993f88c0715be09bb19aee2e267178f2595462ee6ee91cb4942"}
Sep 10 21:46:13.219140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:13.218766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerStarted","Data":"2af256058e8eaa88b8aa86f53cdf53cd24a22cc14a28418a0e4abe77c57a4a24"}
Sep 10 21:46:13.239353 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:13.239307    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podStartSLOduration=1.239291652 podStartE2EDuration="1.239291652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:46:13.238313568 +0000 UTC m=+1609.733137105" watchObservedRunningTime="2026-09-10 21:46:13.239291652 +0000 UTC m=+1609.734115190"
Sep 10 21:46:13.351079 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:13.350469    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:13.361416 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:13.361351    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:14.223257 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:14.223225    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:14.224378 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:14.224361    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:15.158236 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:15.158196    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:15.258696 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:15.258671    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:16.421836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:16.421808    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:16.528172 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:16.528119    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:16.666241 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:16.666191    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:17.071636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.071602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:17.168040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.168010    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:17.246642 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.246612    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:17.249996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.249980    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.253167 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.253134    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-bf96d8\""
Sep 10 21:46:17.253304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.253192    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-bf96d8-dockercfg-tbs9x\""
Sep 10 21:46:17.264043 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.264021    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:17.342348 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.342273    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:17.345678 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.345656    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.359748 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.359721    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:17.430471 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.430445    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8qd4\" (UniqueName: \"kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.430848 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.430572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.531161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.531120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skmxx\" (UniqueName: \"kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.531319 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.531219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.531319 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.531246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8qd4\" (UniqueName: \"kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.531319 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.531263    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.531898 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.531880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.553697 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.553668    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8qd4\" (UniqueName: \"kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4\") pod \"payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.560519 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.560496    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:17.633123 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.632719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.633123 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.632770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skmxx\" (UniqueName: \"kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.633517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.633496    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.641046 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.641019    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skmxx\" (UniqueName: \"kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx\") pod \"payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.655835 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.655813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:17.836672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.836646    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:17.838558 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:46:17.838529    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a678_08d8_4710_aa2c_6b539f06e4f1.slice/crio-4cccc75fdec04a1e9f41a692990d48995ec8ac964b647010889b43ac15e173b6 WatchSource:0}: Error finding container 4cccc75fdec04a1e9f41a692990d48995ec8ac964b647010889b43ac15e173b6: Status 404 returned error can't find the container with id 4cccc75fdec04a1e9f41a692990d48995ec8ac964b647010889b43ac15e173b6
Sep 10 21:46:17.937267 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:17.937237    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:17.940367 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:46:17.940330    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod710d73d2_0fc7_4607_9494_c9184685eb1b.slice/crio-0aacd7df5dd8b468ec8f0e01521177023a87aaf8aba8e842d0faeea956df88ac WatchSource:0}: Error finding container 0aacd7df5dd8b468ec8f0e01521177023a87aaf8aba8e842d0faeea956df88ac: Status 404 returned error can't find the container with id 0aacd7df5dd8b468ec8f0e01521177023a87aaf8aba8e842d0faeea956df88ac
Sep 10 21:46:18.244666 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.244501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" event={"ID":"3961a678-08d8-4710-aa2c-6b539f06e4f1","Type":"ContainerStarted","Data":"69d0ee90173cb8229aa7bcbc18cae3c3da3e2bb9e9184bf4b61e2e63ab4af09f"}
Sep 10 21:46:18.244666 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.244567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" event={"ID":"3961a678-08d8-4710-aa2c-6b539f06e4f1","Type":"ContainerStarted","Data":"4cccc75fdec04a1e9f41a692990d48995ec8ac964b647010889b43ac15e173b6"}
Sep 10 21:46:18.244935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.244751    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:18.246407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.246383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" event={"ID":"710d73d2-0fc7-4607-9494-c9184685eb1b","Type":"ContainerStarted","Data":"3a2b7e04c9edb692d7ef5bfd2503c8bea5a5ec3dbc623e2024efaf0bc9f569ee"}
Sep 10 21:46:18.246561 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.246418    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" event={"ID":"710d73d2-0fc7-4607-9494-c9184685eb1b","Type":"ContainerStarted","Data":"0aacd7df5dd8b468ec8f0e01521177023a87aaf8aba8e842d0faeea956df88ac"}
Sep 10 21:46:18.246561 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.246526    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:18.261157 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.261106    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" podStartSLOduration=1.261089854 podStartE2EDuration="1.261089854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:46:18.259799108 +0000 UTC m=+1614.754622648" watchObservedRunningTime="2026-09-10 21:46:18.261089854 +0000 UTC m=+1614.755913396"
Sep 10 21:46:18.277062 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:18.277014    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" podStartSLOduration=1.277000501 podStartE2EDuration="1.277000501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:46:18.274925283 +0000 UTC m=+1614.769748822" watchObservedRunningTime="2026-09-10 21:46:18.277000501 +0000 UTC m=+1614.771824039"
Sep 10 21:46:24.194994 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:24.194961    2579 scope.go:117] "RemoveContainer" containerID="65a5b52ff76363879ed8a66209d87f30c4284d85d3b5d0d9b44825c864a8a8cd"
Sep 10 21:46:29.255475 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:29.255443    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:29.256503 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:29.256475    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:35.337287 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.337204    2579 generic.go:356] "Generic (PLEG): container finished" podID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerID="52348fea5c927993f88c0715be09bb19aee2e267178f2595462ee6ee91cb4942" exitCode=0
Sep 10 21:46:35.337686 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.337279    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerDied","Data":"52348fea5c927993f88c0715be09bb19aee2e267178f2595462ee6ee91cb4942"}
Sep 10 21:46:35.337735 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.337685    2579 scope.go:117] "RemoveContainer" containerID="52348fea5c927993f88c0715be09bb19aee2e267178f2595462ee6ee91cb4942"
Sep 10 21:46:35.339056 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.339026    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:35.339142 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.339097    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:35.339142 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:35.339127    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 10 21:46:36.350217 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:36.350174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerStarted","Data":"3e9a846ff9e0d3507eb9e4bfe617567b981abf1bb74deb88e9890196536904cc"}
Sep 10 21:46:37.349951 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:37.349911    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:37.354691 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:37.350221    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:46:37.355325 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:37.355300    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:46:38.350857 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:38.350814    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:46:38.351074 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:38.350892    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:46:39.350803 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:39.350740    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:46:39.351413 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:39.351378    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:46:40.159335 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.159276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:40.350239 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.350201    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:46:40.350517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.350487    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:46:40.830806 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.830757    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"]
Sep 10 21:46:40.833952 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.833936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:40.837974 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.837943    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:46:40.838243 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.838223    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:46:40.841961 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.841934    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"]
Sep 10 21:46:40.948805 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:40.942971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm\" (UID: \"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:41.043641 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.043593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm\" (UID: \"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:41.044704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.044676    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm\" (UID: \"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:41.154881 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.154759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:41.355804 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.352139    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 21:46:41.355804 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.352202    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 21:46:41.429491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:41.429464    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"]
Sep 10 21:46:41.431360 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:46:41.431331    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f59cc44_deaf_46c2_89f0_1d1ca5d8e259.slice/crio-34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8 WatchSource:0}: Error finding container 34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8: Status 404 returned error can't find the container with id 34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8
Sep 10 21:46:42.353971 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.353880    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:42.354442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.354148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:42.354750 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.354733    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:42.384013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.383985    2579 generic.go:356] "Generic (PLEG): container finished" podID="9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" containerID="3c230bcfca73449aac199b95bf1b7ccd4ae860d3310e7f3797507a0fcac36954" exitCode=0
Sep 10 21:46:42.384165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.384071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm" event={"ID":"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259","Type":"ContainerDied","Data":"3c230bcfca73449aac199b95bf1b7ccd4ae860d3310e7f3797507a0fcac36954"}
Sep 10 21:46:42.384165 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:42.384120    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm" event={"ID":"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259","Type":"ContainerStarted","Data":"34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8"}
Sep 10 21:46:43.514107 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:43.514081    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:43.578161 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:43.578129    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca\") pod \"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259\" (UID: \"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259\") "
Sep 10 21:46:43.578468 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:43.578444    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" (UID: "9f59cc44-deaf-46c2-89f0-1d1ca5d8e259"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:46:43.679333 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:43.679271    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:44.395115 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:44.395089    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"
Sep 10 21:46:44.395281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:44.395083    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm" event={"ID":"9f59cc44-deaf-46c2-89f0-1d1ca5d8e259","Type":"ContainerDied","Data":"34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8"}
Sep 10 21:46:44.395281 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:44.395249    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34b6dcfcc9805a4efb3059e32ed6b23faef84cc1f5ae150b1c654032593cb4d8"
Sep 10 21:46:47.091573 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.091547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.157547 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.157519    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:47.157757 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.157727    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" podUID="710d73d2-0fc7-4607-9494-c9184685eb1b" containerName="payload-processing" containerID="cri-o://3a2b7e04c9edb692d7ef5bfd2503c8bea5a5ec3dbc623e2024efaf0bc9f569ee" gracePeriod=30
Sep 10 21:46:47.174958 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.174932    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:47.175170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.175150    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" podUID="3961a678-08d8-4710-aa2c-6b539f06e4f1" containerName="payload-pre-processing" containerID="cri-o://69d0ee90173cb8229aa7bcbc18cae3c3da3e2bb9e9184bf4b61e2e63ab4af09f" gracePeriod=30
Sep 10 21:46:47.262073 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.262040    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.414168 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.414086    2579 generic.go:356] "Generic (PLEG): container finished" podID="3961a678-08d8-4710-aa2c-6b539f06e4f1" containerID="69d0ee90173cb8229aa7bcbc18cae3c3da3e2bb9e9184bf4b61e2e63ab4af09f" exitCode=0
Sep 10 21:46:47.414339 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.414243    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" event={"ID":"3961a678-08d8-4710-aa2c-6b539f06e4f1","Type":"ContainerDied","Data":"69d0ee90173cb8229aa7bcbc18cae3c3da3e2bb9e9184bf4b61e2e63ab4af09f"}
Sep 10 21:46:47.416312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.416286    2579 generic.go:356] "Generic (PLEG): container finished" podID="710d73d2-0fc7-4607-9494-c9184685eb1b" containerID="3a2b7e04c9edb692d7ef5bfd2503c8bea5a5ec3dbc623e2024efaf0bc9f569ee" exitCode=0
Sep 10 21:46:47.416454 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.416329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" event={"ID":"710d73d2-0fc7-4607-9494-c9184685eb1b","Type":"ContainerDied","Data":"3a2b7e04c9edb692d7ef5bfd2503c8bea5a5ec3dbc623e2024efaf0bc9f569ee"}
Sep 10 21:46:47.420707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.420681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.464317 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.464295    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:47.473208 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.473185    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:47.489730 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.489710    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"]
Sep 10 21:46:47.493984 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.493963    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-bf96d8-jdcnm"]
Sep 10 21:46:47.496304 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.496276    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.574055 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.574025    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.617151 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617122    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume\") pod \"710d73d2-0fc7-4607-9494-c9184685eb1b\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") "
Sep 10 21:46:47.617312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617184    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8qd4\" (UniqueName: \"kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4\") pod \"3961a678-08d8-4710-aa2c-6b539f06e4f1\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") "
Sep 10 21:46:47.617312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617212    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume\") pod \"3961a678-08d8-4710-aa2c-6b539f06e4f1\" (UID: \"3961a678-08d8-4710-aa2c-6b539f06e4f1\") "
Sep 10 21:46:47.617312 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skmxx\" (UniqueName: \"kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx\") pod \"710d73d2-0fc7-4607-9494-c9184685eb1b\" (UID: \"710d73d2-0fc7-4607-9494-c9184685eb1b\") "
Sep 10 21:46:47.617602 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617578    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "710d73d2-0fc7-4607-9494-c9184685eb1b" (UID: "710d73d2-0fc7-4607-9494-c9184685eb1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:46:47.617655 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.617633    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3961a678-08d8-4710-aa2c-6b539f06e4f1" (UID: "3961a678-08d8-4710-aa2c-6b539f06e4f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:46:47.619517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.619490    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx" (OuterVolumeSpecName: "kube-api-access-skmxx") pod "710d73d2-0fc7-4607-9494-c9184685eb1b" (UID: "710d73d2-0fc7-4607-9494-c9184685eb1b"). InnerVolumeSpecName "kube-api-access-skmxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:46:47.619579 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.619523    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4" (OuterVolumeSpecName: "kube-api-access-h8qd4") pod "3961a678-08d8-4710-aa2c-6b539f06e4f1" (UID: "3961a678-08d8-4710-aa2c-6b539f06e4f1"). InnerVolumeSpecName "kube-api-access-h8qd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:46:47.718472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.718368    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/710d73d2-0fc7-4607-9494-c9184685eb1b-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:47.718472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.718403    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8qd4\" (UniqueName: \"kubernetes.io/projected/3961a678-08d8-4710-aa2c-6b539f06e4f1-kube-api-access-h8qd4\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:47.718472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.718416    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3961a678-08d8-4710-aa2c-6b539f06e4f1-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:47.718472 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.718431    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skmxx\" (UniqueName: \"kubernetes.io/projected/710d73d2-0fc7-4607-9494-c9184685eb1b-kube-api-access-skmxx\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:47.852173 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.852129    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:47.956128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:47.956086    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" path="/var/lib/kubelet/pods/9f59cc44-deaf-46c2-89f0-1d1ca5d8e259/volumes"
Sep 10 21:46:48.423210 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.423117    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"
Sep 10 21:46:48.423210 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.423163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq" event={"ID":"710d73d2-0fc7-4607-9494-c9184685eb1b","Type":"ContainerDied","Data":"0aacd7df5dd8b468ec8f0e01521177023a87aaf8aba8e842d0faeea956df88ac"}
Sep 10 21:46:48.423731 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.423238    2579 scope.go:117] "RemoveContainer" containerID="3a2b7e04c9edb692d7ef5bfd2503c8bea5a5ec3dbc623e2024efaf0bc9f569ee"
Sep 10 21:46:48.424716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.424551    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx" event={"ID":"3961a678-08d8-4710-aa2c-6b539f06e4f1","Type":"ContainerDied","Data":"4cccc75fdec04a1e9f41a692990d48995ec8ac964b647010889b43ac15e173b6"}
Sep 10 21:46:48.424716 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.424594    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"
Sep 10 21:46:48.432651 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.432635    2579 scope.go:117] "RemoveContainer" containerID="69d0ee90173cb8229aa7bcbc18cae3c3da3e2bb9e9184bf4b61e2e63ab4af09f"
Sep 10 21:46:48.442505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.442483    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:48.448533 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.448511    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-bf96d8-58f9b7d7db-fb7dx"]
Sep 10 21:46:48.458474 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.458452    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:48.465391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:48.465369    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-bf96d8-76665b48c7-9jptq"]
Sep 10 21:46:49.956133 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:49.956087    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3961a678-08d8-4710-aa2c-6b539f06e4f1" path="/var/lib/kubelet/pods/3961a678-08d8-4710-aa2c-6b539f06e4f1/volumes"
Sep 10 21:46:49.956632 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:49.956590    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="710d73d2-0fc7-4607-9494-c9184685eb1b" path="/var/lib/kubelet/pods/710d73d2-0fc7-4607-9494-c9184685eb1b/volumes"
Sep 10 21:46:54.033086 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:54.033057    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:54.113185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:54.113151    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:54.278795 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:54.278753    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:46:54.279063 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:54.279018    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" containerID="cri-o://3e9a846ff9e0d3507eb9e4bfe617567b981abf1bb74deb88e9890196536904cc" gracePeriod=30
Sep 10 21:46:54.326344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:54.326281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:46:57.356724 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:57.356684    2579 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:46:57.357105 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:57.356742    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:46:59.480852 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.480819    2579 generic.go:356] "Generic (PLEG): container finished" podID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerID="3e9a846ff9e0d3507eb9e4bfe617567b981abf1bb74deb88e9890196536904cc" exitCode=0
Sep 10 21:46:59.481182 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.480891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerDied","Data":"3e9a846ff9e0d3507eb9e4bfe617567b981abf1bb74deb88e9890196536904cc"}
Sep 10 21:46:59.481182 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.480952    2579 scope.go:117] "RemoveContainer" containerID="52348fea5c927993f88c0715be09bb19aee2e267178f2595462ee6ee91cb4942"
Sep 10 21:46:59.544407 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.544354    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:46:59.623655 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623622    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.623890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.623890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlj5k\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.623890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623763    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.623890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623829    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.623890 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623873    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.624153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623903    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.624153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623928    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.624153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.623953    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket\") pod \"83f77a28-238d-41de-8ccd-7fb8ef98468b\" (UID: \"83f77a28-238d-41de-8ccd-7fb8ef98468b\") "
Sep 10 21:46:59.624153 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624120    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data" (OuterVolumeSpecName: "istio-data") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:46:59.624344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624275    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.624344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624279    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:46:59.624344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624307    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:46:59.624344 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624325    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:46:59.624555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.624374    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:46:59.626103 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.626083    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:46:59.626365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.626346    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token" (OuterVolumeSpecName: "istio-token") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:46:59.626426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.626382    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:46:59.626426 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.626383    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k" (OuterVolumeSpecName: "kube-api-access-vlj5k") pod "83f77a28-238d-41de-8ccd-7fb8ef98468b" (UID: "83f77a28-238d-41de-8ccd-7fb8ef98468b"). InnerVolumeSpecName "kube-api-access-vlj5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:46:59.725359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725331    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725355    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725369    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlj5k\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-kube-api-access-vlj5k\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725381    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725393    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f77a28-238d-41de-8ccd-7fb8ef98468b-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725406    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725417    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f77a28-238d-41de-8ccd-7fb8ef98468b-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:46:59.725505 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:46:59.725428    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f77a28-238d-41de-8ccd-7fb8ef98468b-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:47:00.487613 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:00.487582    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"
Sep 10 21:47:00.488072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:00.487612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc" event={"ID":"83f77a28-238d-41de-8ccd-7fb8ef98468b","Type":"ContainerDied","Data":"2af256058e8eaa88b8aa86f53cdf53cd24a22cc14a28418a0e4abe77c57a4a24"}
Sep 10 21:47:00.488072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:00.487697    2579 scope.go:117] "RemoveContainer" containerID="3e9a846ff9e0d3507eb9e4bfe617567b981abf1bb74deb88e9890196536904cc"
Sep 10 21:47:00.504551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:00.504526    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:47:00.509494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:00.509474    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-bf96d8-openshift-default-f57cd67c8-hf7dc"]
Sep 10 21:47:01.955387 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:01.955305    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" path="/var/lib/kubelet/pods/83f77a28-238d-41de-8ccd-7fb8ef98468b/volumes"
Sep 10 21:47:15.372517 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:15.372482    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:16.953331 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:16.953298    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:17.994006 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.993974    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"]
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994287    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994298    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994306    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="710d73d2-0fc7-4607-9494-c9184685eb1b" containerName="payload-processing"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994311    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="710d73d2-0fc7-4607-9494-c9184685eb1b" containerName="payload-processing"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994318    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3961a678-08d8-4710-aa2c-6b539f06e4f1" containerName="payload-pre-processing"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994324    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3961a678-08d8-4710-aa2c-6b539f06e4f1" containerName="payload-pre-processing"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994336    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" containerName="revoke-keys"
Sep 10 21:47:17.994374 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994341    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" containerName="revoke-keys"
Sep 10 21:47:17.994622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994390    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:47:17.994622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994397    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f59cc44-deaf-46c2-89f0-1d1ca5d8e259" containerName="revoke-keys"
Sep 10 21:47:17.994622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994406    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="710d73d2-0fc7-4607-9494-c9184685eb1b" containerName="payload-processing"
Sep 10 21:47:17.994622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994413    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:47:17.994622 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.994419    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3961a678-08d8-4710-aa2c-6b539f06e4f1" containerName="payload-pre-processing"
Sep 10 21:47:17.998712 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:17.998696    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:18.001106 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.001086    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:47:18.001249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.001149    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:47:18.006202 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.005968    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"]
Sep 10 21:47:18.087164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.087138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp\" (UID: \"a49507a6-c80b-4a76-94f8-117a6da90813\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:18.188004 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.187974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp\" (UID: \"a49507a6-c80b-4a76-94f8-117a6da90813\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:18.188554 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.188536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp\" (UID: \"a49507a6-c80b-4a76-94f8-117a6da90813\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:18.222931 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.222901    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:18.310188 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.310159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:18.579195 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:18.579103    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"]
Sep 10 21:47:18.581943 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:47:18.581918    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49507a6_c80b_4a76_94f8_117a6da90813.slice/crio-892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b WatchSource:0}: Error finding container 892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b: Status 404 returned error can't find the container with id 892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b
Sep 10 21:47:19.583633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:19.583594    2579 generic.go:356] "Generic (PLEG): container finished" podID="a49507a6-c80b-4a76-94f8-117a6da90813" containerID="237427caf5a2d71c341b80ddbdcd998fa790ade7d16ddeb4436ec1300990b5d8" exitCode=0
Sep 10 21:47:19.584028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:19.583639    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp" event={"ID":"a49507a6-c80b-4a76-94f8-117a6da90813","Type":"ContainerDied","Data":"237427caf5a2d71c341b80ddbdcd998fa790ade7d16ddeb4436ec1300990b5d8"}
Sep 10 21:47:19.584028 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:19.583689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp" event={"ID":"a49507a6-c80b-4a76-94f8-117a6da90813","Type":"ContainerStarted","Data":"892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b"}
Sep 10 21:47:20.727192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:20.727168    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:20.811391 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:20.811364    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca\") pod \"a49507a6-c80b-4a76-94f8-117a6da90813\" (UID: \"a49507a6-c80b-4a76-94f8-117a6da90813\") "
Sep 10 21:47:20.811744 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:20.811722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a49507a6-c80b-4a76-94f8-117a6da90813" (UID: "a49507a6-c80b-4a76-94f8-117a6da90813"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:47:20.912383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:20.912332    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a49507a6-c80b-4a76-94f8-117a6da90813-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:47:21.596274 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:21.596248    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"
Sep 10 21:47:21.596436 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:21.596246    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp" event={"ID":"a49507a6-c80b-4a76-94f8-117a6da90813","Type":"ContainerDied","Data":"892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b"}
Sep 10 21:47:21.596436 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:21.596370    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="892c7771847ba19532d08286bcadbdcf064e26a79b8da03e15804493dceab69b"
Sep 10 21:47:33.549772 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.549740    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"]
Sep 10 21:47:33.551470 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.551437    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-8291ef-29817945-w85mw"]
Sep 10 21:47:33.552071 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.552048    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:33.627192 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.627163    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:33.699680 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.699649    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:33.748754 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.748731    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"]
Sep 10 21:47:33.752768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.752746    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8291ef-fsnnp"]
Sep 10 21:47:33.770244 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.770220    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:33.955576 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.955481    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a49507a6-c80b-4a76-94f8-117a6da90813" path="/var/lib/kubelet/pods/a49507a6-c80b-4a76-94f8-117a6da90813/volumes"
Sep 10 21:47:33.955903 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:33.955881    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed63665f-ffe0-4afa-a625-ab0613099c28" path="/var/lib/kubelet/pods/ed63665f-ffe0-4afa-a625-ab0613099c28/volumes"
Sep 10 21:47:34.130893 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:34.130863    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:39.144185 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:39.144146    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:44.086335 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:44.086300    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:52.424647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:52.424566    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:47:53.047923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:47:53.047884    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:15.425528 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:15.425498    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:16.349465 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:16.349433    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:21.903647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:21.903612    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:21.971380 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:21.971348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:22.169116 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.169029    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:48:22.169297 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.169270    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" containerID="cri-o://1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f" gracePeriod=30
Sep 10 21:48:22.181628 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.181605    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:22.529636 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.529602    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"]
Sep 10 21:48:22.530051 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.530031    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a49507a6-c80b-4a76-94f8-117a6da90813" containerName="revoke-keys"
Sep 10 21:48:22.530145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.530053    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a49507a6-c80b-4a76-94f8-117a6da90813" containerName="revoke-keys"
Sep 10 21:48:22.530145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.530075    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:48:22.530145 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.530083    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f77a28-238d-41de-8ccd-7fb8ef98468b" containerName="istio-proxy"
Sep 10 21:48:22.530302 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.530167    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a49507a6-c80b-4a76-94f8-117a6da90813" containerName="revoke-keys"
Sep 10 21:48:22.533125 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.533105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:22.535275 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.535256    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:48:22.535379 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.535359    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:48:22.540852 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.540397    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"]
Sep 10 21:48:22.620050 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.620028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5\" (UID: \"3961a849-587b-4d3a-b854-9581502738f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:22.721410 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.721384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5\" (UID: \"3961a849-587b-4d3a-b854-9581502738f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:22.721991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.721975    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5\" (UID: \"3961a849-587b-4d3a-b854-9581502738f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:22.726802 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.726765    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:22.845090 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:22.845022    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:23.115075 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:23.115060    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"]
Sep 10 21:48:23.117436 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:48:23.117406    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9 WatchSource:0}: Error finding container 82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9: Status 404 returned error can't find the container with id 82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9
Sep 10 21:48:23.909359 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:23.909333    2579 generic.go:356] "Generic (PLEG): container finished" podID="3961a849-587b-4d3a-b854-9581502738f1" containerID="0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0" exitCode=0
Sep 10 21:48:23.909514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:23.909423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5" event={"ID":"3961a849-587b-4d3a-b854-9581502738f1","Type":"ContainerDied","Data":"0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0"}
Sep 10 21:48:23.909514 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:23.909475    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5" event={"ID":"3961a849-587b-4d3a-b854-9581502738f1","Type":"ContainerStarted","Data":"82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9"}
Sep 10 21:48:24.172617 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:24.172564    2579 patch_prober.go:28] interesting pod/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:48:24.172992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:24.172641    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:48:25.047276 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.047252    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:25.141883 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.141853    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca\") pod \"3961a849-587b-4d3a-b854-9581502738f1\" (UID: \"3961a849-587b-4d3a-b854-9581502738f1\") "
Sep 10 21:48:25.142225 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.142200    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3961a849-587b-4d3a-b854-9581502738f1" (UID: "3961a849-587b-4d3a-b854-9581502738f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:48:25.242871 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.242811    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3961a849-587b-4d3a-b854-9581502738f1-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:25.921652 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.921621    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"
Sep 10 21:48:25.921860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.921625    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5" event={"ID":"3961a849-587b-4d3a-b854-9581502738f1","Type":"ContainerDied","Data":"82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9"}
Sep 10 21:48:25.921860 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:25.921767    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82215a5584885bf11315f2056f4878e4276819cd8263df1d8689138c9f0b73a9"
Sep 10 21:48:27.424369 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.424347    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:48:27.560819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560796    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.560819 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560829    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560864    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560912    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560930    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560971    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561007 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.560991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561038    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j62pz\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz\") pod \"af9ca713-897d-4558-b42a-1087e06de374\" (UID: \"af9ca713-897d-4558-b42a-1087e06de374\") "
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561181    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data" (OuterVolumeSpecName: "istio-data") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561191    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561181    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:48:27.561252 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561204    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:48:27.561551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561317    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af9ca713-897d-4558-b42a-1087e06de374-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.561551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561335    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.561551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561349    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.561551 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561362    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.561717 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.561576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:48:27.563379 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.563352    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:48:27.563506 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.563485    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:48:27.563825 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.563802    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz" (OuterVolumeSpecName: "kube-api-access-j62pz") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "kube-api-access-j62pz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:48:27.563930 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.563831    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token" (OuterVolumeSpecName: "istio-token") pod "af9ca713-897d-4558-b42a-1087e06de374" (UID: "af9ca713-897d-4558-b42a-1087e06de374"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:48:27.662370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.662348    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.662370 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.662366    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af9ca713-897d-4558-b42a-1087e06de374-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.662510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.662375    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af9ca713-897d-4558-b42a-1087e06de374-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.662510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.662385    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.662510 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.662394    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j62pz\" (UniqueName: \"kubernetes.io/projected/af9ca713-897d-4558-b42a-1087e06de374-kube-api-access-j62pz\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:27.933478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.933400    2579 generic.go:356] "Generic (PLEG): container finished" podID="af9ca713-897d-4558-b42a-1087e06de374" containerID="1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f" exitCode=0
Sep 10 21:48:27.933478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.933447    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"
Sep 10 21:48:27.933712 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.933481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerDied","Data":"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"}
Sep 10 21:48:27.933712 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.933532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc" event={"ID":"af9ca713-897d-4558-b42a-1087e06de374","Type":"ContainerDied","Data":"f61cbe6018643c6cf8e6c8398d6dd6ce20955f36447a44d997b8d9c0b51c8704"}
Sep 10 21:48:27.933712 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.933554    2579 scope.go:117] "RemoveContainer" containerID="1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"
Sep 10 21:48:27.943534 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.943518    2579 scope.go:117] "RemoveContainer" containerID="01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"
Sep 10 21:48:27.951474 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.951443    2579 scope.go:117] "RemoveContainer" containerID="1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"
Sep 10 21:48:27.951818 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:27.951798    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f\": container with ID starting with 1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f not found: ID does not exist" containerID="1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"
Sep 10 21:48:27.951905 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.951827    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f"} err="failed to get container status \"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f\": rpc error: code = NotFound desc = could not find container \"1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f\": container with ID starting with 1093e729ab36228afdf6fab611cb557313d430d4a6f7234320f9b3c32b45063f not found: ID does not exist"
Sep 10 21:48:27.951905 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.951847    2579 scope.go:117] "RemoveContainer" containerID="01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"
Sep 10 21:48:27.952095 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:27.952074    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e\": container with ID starting with 01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e not found: ID does not exist" containerID="01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"
Sep 10 21:48:27.952163 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.952104    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e"} err="failed to get container status \"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e\": rpc error: code = NotFound desc = could not find container \"01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e\": container with ID starting with 01576d55cdafa0ea155577986e49ddad9095a9e51237e4b9b87bc1d59b4eec2e not found: ID does not exist"
Sep 10 21:48:27.955412 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.955391    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:48:27.960942 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:27.960920    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-8291ef-openshift-default-7cc45d98d8-rl8wc"]
Sep 10 21:48:28.916292 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:28.916256    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:29.059906 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:29.059866    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:29.060089 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:29.059909    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:29.955745 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:29.955710    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af9ca713-897d-4558-b42a-1087e06de374" path="/var/lib/kubelet/pods/af9ca713-897d-4558-b42a-1087e06de374/volumes"
Sep 10 21:48:33.098798 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.098757    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:33.129372 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.129341    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:48:33.129555 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.129532    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" podUID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" containerName="payload-pre-processing" containerID="cri-o://2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e" gracePeriod=30
Sep 10 21:48:33.197478 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.197448    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:33.309595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.309520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:33.364237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.364146    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"]
Sep 10 21:48:33.375083 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.375052    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-2fbf53-dtwn5"]
Sep 10 21:48:33.397633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.397611    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:48:33.453927 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.453893    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:33.511839 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.511804    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume\") pod \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") "
Sep 10 21:48:33.512030 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.511858    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9fc7\" (UniqueName: \"kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7\") pod \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\" (UID: \"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e\") "
Sep 10 21:48:33.512119 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.512075    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" (UID: "001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:48:33.514203 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.514179    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7" (OuterVolumeSpecName: "kube-api-access-j9fc7") pod "001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" (UID: "001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e"). InnerVolumeSpecName "kube-api-access-j9fc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:48:33.613484 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.613441    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:33.613484 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.613474    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9fc7\" (UniqueName: \"kubernetes.io/projected/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e-kube-api-access-j9fc7\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:33.741719 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.741638    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:33.960789 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.960738    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3961a849-587b-4d3a-b854-9581502738f1" path="/var/lib/kubelet/pods/3961a849-587b-4d3a-b854-9581502738f1/volumes"
Sep 10 21:48:33.966704 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.966675    2579 generic.go:356] "Generic (PLEG): container finished" podID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" containerID="2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e" exitCode=0
Sep 10 21:48:33.966870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.966732    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"
Sep 10 21:48:33.966870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.966757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" event={"ID":"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e","Type":"ContainerDied","Data":"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"}
Sep 10 21:48:33.966870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.966828    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2" event={"ID":"001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e","Type":"ContainerDied","Data":"7060121292be359767f63b1e8fb248bfd1e925e93690047e6dd1db7f1a53c99a"}
Sep 10 21:48:33.966870 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.966851    2579 scope.go:117] "RemoveContainer" containerID="2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"
Sep 10 21:48:33.978045 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.978026    2579 scope.go:117] "RemoveContainer" containerID="2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"
Sep 10 21:48:33.978352 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:33.978330    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e\": container with ID starting with 2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e not found: ID does not exist" containerID="2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"
Sep 10 21:48:33.978400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.978364    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e"} err="failed to get container status \"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e\": rpc error: code = NotFound desc = could not find container \"2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e\": container with ID starting with 2c501055a11b4f262530dab8672b501ed411c995f2362dc17221837dac37944e not found: ID does not exist"
Sep 10 21:48:33.988137 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.988108    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:48:33.993135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:33.993081    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2fbf53-6d9fb88d8b-6dcn2"]
Sep 10 21:48:35.346630 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:35.346596    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:35.955706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:35.955667    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" path="/var/lib/kubelet/pods/001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e/volumes"
Sep 10 21:48:38.926875 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:38.926840    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:39.586554 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:39.586527    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:39.657571 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:39.657536    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:39.907406 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:39.907328    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:40.199557 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.199468    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"]
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.199961    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" containerName="payload-pre-processing"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.199979    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" containerName="payload-pre-processing"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.199991    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.199999    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200019    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200027    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200036    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3961a849-587b-4d3a-b854-9581502738f1" containerName="revoke-keys"
Sep 10 21:48:40.200040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200043    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3961a849-587b-4d3a-b854-9581502738f1" containerName="revoke-keys"
Sep 10 21:48:40.200445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200127    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:48:40.200445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200140    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="001e129a-f2f8-47a4-89a8-7f5fc6c8aa1e" containerName="payload-pre-processing"
Sep 10 21:48:40.200445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.200151    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3961a849-587b-4d3a-b854-9581502738f1" containerName="revoke-keys"
Sep 10 21:48:40.210279 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.210253    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:40.212082 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.212059    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"]
Sep 10 21:48:40.212544 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.212521    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 21:48:40.212706 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.212685    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-7wg7l\""
Sep 10 21:48:40.263293 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.263268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd\" (UID: \"3d917584-197f-45cc-8586-4bdea2c623e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:40.343742 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.343721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:40.363980 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.363957    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd\" (UID: \"3d917584-197f-45cc-8586-4bdea2c623e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:40.364481 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.364464    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd\" (UID: \"3d917584-197f-45cc-8586-4bdea2c623e1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:40.522083 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.522056    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:40.791633 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:40.791611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"]
Sep 10 21:48:40.793920 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:48:40.793895    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d917584_197f_45cc_8586_4bdea2c623e1.slice/crio-e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d WatchSource:0}: Error finding container e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d: Status 404 returned error can't find the container with id e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d
Sep 10 21:48:41.001196 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:41.001166    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd" event={"ID":"3d917584-197f-45cc-8586-4bdea2c623e1","Type":"ContainerStarted","Data":"a831e05b8f60d26a14c638c95b4585fbbd254dd661be605fcf758fab1ff13c9b"}
Sep 10 21:48:41.001288 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:41.001217    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd" event={"ID":"3d917584-197f-45cc-8586-4bdea2c623e1","Type":"ContainerStarted","Data":"e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d"}
Sep 10 21:48:41.015475 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:41.015390    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd" podStartSLOduration=1.015374026 podStartE2EDuration="1.015374026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 21:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:48:41.014903173 +0000 UTC m=+1757.509726714" watchObservedRunningTime="2026-09-10 21:48:41.015374026 +0000 UTC m=+1757.510197568"
Sep 10 21:48:42.007456 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:42.007419    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d917584-197f-45cc-8586-4bdea2c623e1" containerID="a831e05b8f60d26a14c638c95b4585fbbd254dd661be605fcf758fab1ff13c9b" exitCode=0
Sep 10 21:48:42.007844 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:42.007508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd" event={"ID":"3d917584-197f-45cc-8586-4bdea2c623e1","Type":"ContainerDied","Data":"a831e05b8f60d26a14c638c95b4585fbbd254dd661be605fcf758fab1ff13c9b"}
Sep 10 21:48:43.138114 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:43.138092    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:43.186132 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:43.186108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca\") pod \"3d917584-197f-45cc-8586-4bdea2c623e1\" (UID: \"3d917584-197f-45cc-8586-4bdea2c623e1\") "
Sep 10 21:48:43.186424 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:43.186403    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d917584-197f-45cc-8586-4bdea2c623e1" (UID: "3d917584-197f-45cc-8586-4bdea2c623e1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:48:43.287249 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:43.287225    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d917584-197f-45cc-8586-4bdea2c623e1-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:44.018448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:44.018406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd" event={"ID":"3d917584-197f-45cc-8586-4bdea2c623e1","Type":"ContainerDied","Data":"e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d"}
Sep 10 21:48:44.018448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:44.018429    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"
Sep 10 21:48:44.018619 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:44.018456    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4d07fa73e4bee391abfa563be86c58b83ff1578d77e11930c06270e8d1e2a0d"
Sep 10 21:48:45.830217 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:45.830184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:47.958386 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:47.958347    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:48.598341 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:48.598315    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:48.968847 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:48.968758    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:50.319270 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:50.319233    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:48:50.714219 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.714132    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:50.745140 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.745110    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:48:50.745381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.745349    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" podUID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" containerName="payload-processing" containerID="cri-o://e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6" gracePeriod=30
Sep 10 21:48:50.817013 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.816984    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:50.951295 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.951269    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:50.974040 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.974018    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"]
Sep 10 21:48:50.984638 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.984604    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-2d9e68-fpbcd"]
Sep 10 21:48:50.990911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:50.990893    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:48:51.044237 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.044210    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume\") pod \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") "
Sep 10 21:48:51.044390 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.044246    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdhbk\" (UniqueName: \"kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk\") pod \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\" (UID: \"3332dbb5-87ef-4611-8e73-5416fcb6eea0\") "
Sep 10 21:48:51.044644 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.044620    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3332dbb5-87ef-4611-8e73-5416fcb6eea0" (UID: "3332dbb5-87ef-4611-8e73-5416fcb6eea0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:48:51.046598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.046576    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk" (OuterVolumeSpecName: "kube-api-access-fdhbk") pod "3332dbb5-87ef-4611-8e73-5416fcb6eea0" (UID: "3332dbb5-87ef-4611-8e73-5416fcb6eea0"). InnerVolumeSpecName "kube-api-access-fdhbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:48:51.054163 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.054136    2579 generic.go:356] "Generic (PLEG): container finished" podID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" containerID="e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6" exitCode=0
Sep 10 21:48:51.054289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.054183    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"
Sep 10 21:48:51.054289 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.054183    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" event={"ID":"3332dbb5-87ef-4611-8e73-5416fcb6eea0","Type":"ContainerDied","Data":"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"}
Sep 10 21:48:51.054384 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.054287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx" event={"ID":"3332dbb5-87ef-4611-8e73-5416fcb6eea0","Type":"ContainerDied","Data":"b5af199302ce98ec9e4db9cb11a545e5f1c8d2d1e2405a546f45b3a4cb759f3b"}
Sep 10 21:48:51.054384 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.054317    2579 scope.go:117] "RemoveContainer" containerID="e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"
Sep 10 21:48:51.063701 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.063678    2579 scope.go:117] "RemoveContainer" containerID="e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"
Sep 10 21:48:51.064020 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:51.064001    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6\": container with ID starting with e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6 not found: ID does not exist" containerID="e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"
Sep 10 21:48:51.064101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.064028    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6"} err="failed to get container status \"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6\": rpc error: code = NotFound desc = could not find container \"e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6\": container with ID starting with e959549b4e3b63d9f53227564585b62b885846a61e45cd7cd97b442fd8b87df6 not found: ID does not exist"
Sep 10 21:48:51.086873 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.086848    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:48:51.098058 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.098028    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-2d9e68-775c4d4859-scnbx"]
Sep 10 21:48:51.115296 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.115274    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:51.145337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.145313    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3332dbb5-87ef-4611-8e73-5416fcb6eea0-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:51.145337 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.145335    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdhbk\" (UniqueName: \"kubernetes.io/projected/3332dbb5-87ef-4611-8e73-5416fcb6eea0-kube-api-access-fdhbk\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:48:51.371833 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.371804    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:51.960672 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.960642    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" path="/var/lib/kubelet/pods/3332dbb5-87ef-4611-8e73-5416fcb6eea0/volumes"
Sep 10 21:48:51.961023 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:51.961007    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d917584-197f-45cc-8586-4bdea2c623e1" path="/var/lib/kubelet/pods/3d917584-197f-45cc-8586-4bdea2c623e1/volumes"
Sep 10 21:48:57.477497 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:57.477444    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:57.544465 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:57.544410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:57.738895 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:57.738821    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:58.340358 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:48:58.340311    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:48:59.007265 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:48:59.007230    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:49:05.361173 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:05.361141    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:49:08.551259 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.551228    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:08.555707 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.555682    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"]
Sep 10 21:49:08.558089 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.558066    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-67d5e2-29817945-kq4tf"]
Sep 10 21:49:08.597365 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.597335    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:49:08.597647 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.597588    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" podUID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" containerName="payload-processing" containerID="cri-o://d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef" gracePeriod=30
Sep 10 21:49:08.605996 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.605973    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:49:08.606179 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.606160    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" podUID="6049be15-1e81-4fbe-a677-db9c2428eead" containerName="payload-pre-processing" containerID="cri-o://4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738" gracePeriod=30
Sep 10 21:49:08.641054 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.641027    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:08.713414 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.713387    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:08.781873 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.781844    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:08.850317 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.850250    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:08.874101 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.874080    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:49:08.876767 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.876750    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:49:08.990696 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.990664    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptwzb\" (UniqueName: \"kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb\") pod \"6049be15-1e81-4fbe-a677-db9c2428eead\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") "
Sep 10 21:49:08.990911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.990706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume\") pod \"6049be15-1e81-4fbe-a677-db9c2428eead\" (UID: \"6049be15-1e81-4fbe-a677-db9c2428eead\") "
Sep 10 21:49:08.990911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.990752    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume\") pod \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") "
Sep 10 21:49:08.990911 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.990807    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qp2c\" (UniqueName: \"kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c\") pod \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\" (UID: \"24d9c529-cab8-48e4-b1ea-3c910fe6a579\") "
Sep 10 21:49:08.991164 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.991138    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6049be15-1e81-4fbe-a677-db9c2428eead" (UID: "6049be15-1e81-4fbe-a677-db9c2428eead"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:49:08.991220 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.991139    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24d9c529-cab8-48e4-b1ea-3c910fe6a579" (UID: "24d9c529-cab8-48e4-b1ea-3c910fe6a579"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:49:08.993025 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.993004    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c" (OuterVolumeSpecName: "kube-api-access-2qp2c") pod "24d9c529-cab8-48e4-b1ea-3c910fe6a579" (UID: "24d9c529-cab8-48e4-b1ea-3c910fe6a579"). InnerVolumeSpecName "kube-api-access-2qp2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:49:08.993025 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:08.993012    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb" (OuterVolumeSpecName: "kube-api-access-ptwzb") pod "6049be15-1e81-4fbe-a677-db9c2428eead" (UID: "6049be15-1e81-4fbe-a677-db9c2428eead"). InnerVolumeSpecName "kube-api-access-ptwzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:49:09.015495 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:09.015471    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:49:09.091727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.091695    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptwzb\" (UniqueName: \"kubernetes.io/projected/6049be15-1e81-4fbe-a677-db9c2428eead-kube-api-access-ptwzb\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:09.091727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.091729    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6049be15-1e81-4fbe-a677-db9c2428eead-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:09.091876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.091739    2579 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24d9c529-cab8-48e4-b1ea-3c910fe6a579-plugins-config-volume\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:09.091876 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.091749    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qp2c\" (UniqueName: \"kubernetes.io/projected/24d9c529-cab8-48e4-b1ea-3c910fe6a579-kube-api-access-2qp2c\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:09.146305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.146243    2579 generic.go:356] "Generic (PLEG): container finished" podID="6049be15-1e81-4fbe-a677-db9c2428eead" containerID="4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738" exitCode=0
Sep 10 21:49:09.146305 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.146292    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"
Sep 10 21:49:09.146428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.146324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" event={"ID":"6049be15-1e81-4fbe-a677-db9c2428eead","Type":"ContainerDied","Data":"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"}
Sep 10 21:49:09.146428 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.146383    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb" event={"ID":"6049be15-1e81-4fbe-a677-db9c2428eead","Type":"ContainerDied","Data":"93a6070f3bf3d89eb32406629d77600a7dc842f87e0f953471d8884ce2d7d7e5"}
Sep 10 21:49:09.146511 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.146415    2579 scope.go:117] "RemoveContainer" containerID="4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"
Sep 10 21:49:09.147892 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.147871    2579 generic.go:356] "Generic (PLEG): container finished" podID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" containerID="d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef" exitCode=0
Sep 10 21:49:09.147975 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.147894    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" event={"ID":"24d9c529-cab8-48e4-b1ea-3c910fe6a579","Type":"ContainerDied","Data":"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"}
Sep 10 21:49:09.147975 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.147935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4" event={"ID":"24d9c529-cab8-48e4-b1ea-3c910fe6a579","Type":"ContainerDied","Data":"64c20cfa6b618d1a0516c4f6339a7b74340cb0d6e1d222099c6cd0ee01f2a6cc"}
Sep 10 21:49:09.147975 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.147933    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"
Sep 10 21:49:09.155122 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.155100    2579 scope.go:117] "RemoveContainer" containerID="4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"
Sep 10 21:49:09.155480 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:09.155367    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738\": container with ID starting with 4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738 not found: ID does not exist" containerID="4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"
Sep 10 21:49:09.155480 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.155393    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738"} err="failed to get container status \"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738\": rpc error: code = NotFound desc = could not find container \"4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738\": container with ID starting with 4b77e08c0c7ac8ff305721cb8861553a0d205f623824f355c9fe0d97943e5738 not found: ID does not exist"
Sep 10 21:49:09.155480 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.155411    2579 scope.go:117] "RemoveContainer" containerID="d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"
Sep 10 21:49:09.159172 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.159141    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:09.163562 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.163546    2579 scope.go:117] "RemoveContainer" containerID="d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"
Sep 10 21:49:09.163835 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:09.163818    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef\": container with ID starting with d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef not found: ID does not exist" containerID="d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"
Sep 10 21:49:09.163884 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.163841    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef"} err="failed to get container status \"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef\": rpc error: code = NotFound desc = could not find container \"d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef\": container with ID starting with d7a84b2fd7af29ca8888c08040e3d22dc74d413590b70a06d6ec9b7bb13937ef not found: ID does not exist"
Sep 10 21:49:09.171650 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.171619    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:49:09.175422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.175393    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-67d5e2-5875469454-c6lpb"]
Sep 10 21:49:09.184809 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.184788    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:49:09.188938 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.188920    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-67d5e2-5cb54c5cb7-tc2t4"]
Sep 10 21:49:09.956491 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.956457    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" path="/var/lib/kubelet/pods/24d9c529-cab8-48e4-b1ea-3c910fe6a579/volumes"
Sep 10 21:49:09.956888 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.956852    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6049be15-1e81-4fbe-a677-db9c2428eead" path="/var/lib/kubelet/pods/6049be15-1e81-4fbe-a677-db9c2428eead/volumes"
Sep 10 21:49:09.957108 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:09.957088    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9184245d-af57-40d4-825f-9bb4d0536546" path="/var/lib/kubelet/pods/9184245d-af57-40d4-825f-9bb4d0536546/volumes"
Sep 10 21:49:15.255887 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:15.255833    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:15.332548 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:15.332520    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:15.494170 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:15.494086    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:49:15.494699 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:15.494666    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" containerID="cri-o://57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b" gracePeriod=30
Sep 10 21:49:15.517730 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:15.517705    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:16.668016 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:16.667982    2579 patch_prober.go:28] interesting pod/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 21:49:16.668446 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:16.668041    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 21:49:18.182668 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:18.182643    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:49:19.052081 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:19.052050    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:49:20.325759 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:20.325721    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3961a849_587b_4d3a_b854_9581502738f1.slice/crio-conmon-0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0.scope\": RecentStats: unable to find data in memory cache]"
Sep 10 21:49:20.749141 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.749119    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:49:20.895674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895601    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895674 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895641    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895673    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895696    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895721    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895830    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895857    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.895902 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895882    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcljh\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh\") pod \"35b3b44c-20fa-47dd-bffc-f03f1328242e\" (UID: \"35b3b44c-20fa-47dd-bffc-f03f1328242e\") "
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895895    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data" (OuterVolumeSpecName: "istio-data") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.895936    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.896097    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.896124    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-credential-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.896141    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-data\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.896230 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.896173    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:49:20.896709 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.896225    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 21:49:20.898024 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.897991    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 21:49:20.898146 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.898056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 21:49:20.898146 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.898056    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token" (OuterVolumeSpecName: "istio-token") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:49:20.898146 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.898126    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh" (OuterVolumeSpecName: "kube-api-access-pcljh") pod "35b3b44c-20fa-47dd-bffc-f03f1328242e" (UID: "35b3b44c-20fa-47dd-bffc-f03f1328242e"). InnerVolumeSpecName "kube-api-access-pcljh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 21:49:20.996791 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996765    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-podinfo\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996805    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-token\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996815    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-istio-envoy\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996822    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996829    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b3b44c-20fa-47dd-bffc-f03f1328242e-workload-socket\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996837    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b3b44c-20fa-47dd-bffc-f03f1328242e-istiod-ca-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:20.996885 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:20.996846    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcljh\" (UniqueName: \"kubernetes.io/projected/35b3b44c-20fa-47dd-bffc-f03f1328242e-kube-api-access-pcljh\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Sep 10 21:49:21.208800 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.208702    2579 generic.go:356] "Generic (PLEG): container finished" podID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerID="57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b" exitCode=0
Sep 10 21:49:21.208800 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.208745    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"
Sep 10 21:49:21.208992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.208798    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerDied","Data":"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"}
Sep 10 21:49:21.208992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.208849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr" event={"ID":"35b3b44c-20fa-47dd-bffc-f03f1328242e","Type":"ContainerDied","Data":"6d3ae5f516618a2fb134372220c095cc4c23df7c406eed176ec49a398a12926c"}
Sep 10 21:49:21.208992 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.208870    2579 scope.go:117] "RemoveContainer" containerID="57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"
Sep 10 21:49:21.217676 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.217660    2579 scope.go:117] "RemoveContainer" containerID="f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"
Sep 10 21:49:21.225447 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.225434    2579 scope.go:117] "RemoveContainer" containerID="57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"
Sep 10 21:49:21.225714 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:21.225697    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b\": container with ID starting with 57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b not found: ID does not exist" containerID="57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"
Sep 10 21:49:21.225758 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.225723    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b"} err="failed to get container status \"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b\": rpc error: code = NotFound desc = could not find container \"57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b\": container with ID starting with 57712cd3f6fef983f5b047d4ca37a45cc707725fe09162f93dd6137c4969488b not found: ID does not exist"
Sep 10 21:49:21.225758 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.225751    2579 scope.go:117] "RemoveContainer" containerID="f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"
Sep 10 21:49:21.226409 ip-10-0-128-173 kubenswrapper[2579]: E0910 21:49:21.226393    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a\": container with ID starting with f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a not found: ID does not exist" containerID="f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"
Sep 10 21:49:21.226445 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.226417    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a"} err="failed to get container status \"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a\": rpc error: code = NotFound desc = could not find container \"f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a\": container with ID starting with f9e9319ec77551cc64039e71e15a4d9dc8e279294988f5c147b31e56d45d586a not found: ID does not exist"
Sep 10 21:49:21.239228 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.239210    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:49:21.243253 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.243234    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-67d5e2-openshift-default-5c58c74f6d-tdllr"]
Sep 10 21:49:21.955258 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:21.955233    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" path="/var/lib/kubelet/pods/35b3b44c-20fa-47dd-bffc-f03f1328242e/volumes"
Sep 10 21:49:24.245141 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:49:24.245110    2579 scope.go:117] "RemoveContainer" containerID="ef8a6d7d74ef6067e847c867378adac7fe01444db7a2eec95881b86a3de92444"
Sep 10 21:50:04.533945 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:04.533912    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:50:24.285963 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:24.285928    2579 scope.go:117] "RemoveContainer" containerID="f8c591c327ff2488611d506c720ee93c6a708efd5d4732394ca8fc57761a9a2b"
Sep 10 21:50:24.294245 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:24.294229    2579 scope.go:117] "RemoveContainer" containerID="719972ac1f5c3ec1b060a1b7d19a6b6e37e2b0d6c8e8e13f81eec2465d818ac8"
Sep 10 21:50:24.301599 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:24.301575    2579 scope.go:117] "RemoveContainer" containerID="8ebfb66616452bc4a835b90b5228bbc636f101416b57b2385928766c805b37cf"
Sep 10 21:50:51.186580 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:51.186492    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:50:52.788570 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:50:52.788542    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:51:24.324385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:51:24.324354    2579 scope.go:117] "RemoveContainer" containerID="3e35f0bc9b199865c3d100e45c3ab99f7a1b75b25c4cc98a14c7f487d9da2a0c"
Sep 10 21:51:24.332598 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:51:24.332575    2579 scope.go:117] "RemoveContainer" containerID="30f4f985a1d0a89acc15afb1df19f373e59325b52bd221c14584031ef886a999"
Sep 10 21:52:10.691345 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:52:10.691306    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:52:16.496995 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:52:16.496906    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:52:16.553966 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:52:16.553936    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:53:24.359231 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:53:24.359196    2579 scope.go:117] "RemoveContainer" containerID="237427caf5a2d71c341b80ddbdcd998fa790ade7d16ddeb4436ec1300990b5d8"
Sep 10 21:53:24.367954 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:53:24.367927    2579 scope.go:117] "RemoveContainer" containerID="3c230bcfca73449aac199b95bf1b7ccd4ae860d3310e7f3797507a0fcac36954"
Sep 10 21:53:41.830055 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:53:41.830020    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:53:44.901912 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:53:44.901862    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:53:45.002448 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:53:45.002410    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:54:03.425687 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:54:03.425650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:54:21.195680 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:54:21.195650    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:54:22.874535 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:54:22.874499    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:54:24.388277 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:54:24.388248    2579 scope.go:117] "RemoveContainer" containerID="0980989f33e25f30deba119ba462dbef0aa4f8f76deab79b79d90bc7165f08c0"
Sep 10 21:54:46.290027 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:54:46.289998    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:55:08.047684 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:08.047629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:55:15.934513 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:15.934437    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f4574d5b-8b4rc_41238f45-b705-4f93-a8bf-dc695dc28fb0/authorino/0.log"
Sep 10 21:55:21.601089 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:21.601059    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vngxn_dd012855-932e-4c18-a81d-d2162ee5610f/server/0.log"
Sep 10 21:55:21.719759 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:21.719715    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dfmvp_06462169-a019-4ace-a50b-0ee178d060c4/manager/0.log"
Sep 10 21:55:22.166178 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:22.166151    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fc979fdc-sjwrs_5200c0c5-382f-4333-9890-325309b0cd2f/copy-manifests/0.log"
Sep 10 21:55:22.173396 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:22.173369    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fc979fdc-sjwrs_5200c0c5-382f-4333-9890-325309b0cd2f/manager/0.log"
Sep 10 21:55:22.835568 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:22.835531    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29817945-dx5fz_32a7c823-a790-4545-a0c1-c6bdd6360455/cleanup/0.log"
Sep 10 21:55:23.727181 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:23.727146    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f4574d5b-8b4rc_41238f45-b705-4f93-a8bf-dc695dc28fb0/authorino/0.log"
Sep 10 21:55:24.406559 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:24.406526    2579 scope.go:117] "RemoveContainer" containerID="a831e05b8f60d26a14c638c95b4585fbbd254dd661be605fcf758fab1ff13c9b"
Sep 10 21:55:24.804422 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:24.804394    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs_2842bf44-3f01-49e1-b186-c151e13b0778/istio-proxy/0.log"
Sep 10 21:55:24.915364 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:24.915319    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7q5k9_32c71e42-581a-4c01-9c95-12fc34c48b71/discovery/0.log"
Sep 10 21:55:26.049494 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:26.049458    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-x8w7q_f03b47d0-7c75-40b4-87eb-76d10a545043/main/0.log"
Sep 10 21:55:26.158643 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:26.158615    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-959xv_997781d6-f276-432d-9f81-882218a62074/main/0.log"
Sep 10 21:55:26.272831 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:26.272808    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-smsx8_20d07898-0a93-4cf1-b6c1-3489bfed0e4c/main/0.log"
Sep 10 21:55:32.389128 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:32.389092    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:55:32.606400 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:32.606363    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log" limit=1024
Sep 10 21:55:35.418373 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:35.418335    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xqrzz_62848a85-c65b-4a2d-95f0-c7ebaa3a4fc5/global-pull-secret-syncer/0.log"
Sep 10 21:55:35.463768 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:35.463729    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bcwzb_58d0cf93-576a-44e5-9fdc-3d7813d8e372/konnectivity-agent/0.log"
Sep 10 21:55:35.558950 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:35.558902    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-173.ec2.internal_b2d6ebd71b368b317d437cef68944460/haproxy/0.log"
Sep 10 21:55:40.243597 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:40.243561    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9f4574d5b-8b4rc_41238f45-b705-4f93-a8bf-dc695dc28fb0/authorino/0.log"
Sep 10 21:55:42.606099 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:42.606068    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m75sz_b6a7c330-a164-4b38-8639-ce4eeadd7f17/node-exporter/0.log"
Sep 10 21:55:42.626394 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:42.626366    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m75sz_b6a7c330-a164-4b38-8639-ce4eeadd7f17/kube-rbac-proxy/0.log"
Sep 10 21:55:42.646822 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:42.646771    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m75sz_b6a7c330-a164-4b38-8639-ce4eeadd7f17/init-textfile/0.log"
Sep 10 21:55:46.654381 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654333    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"]
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654760    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d917584-197f-45cc-8586-4bdea2c623e1" containerName="revoke-keys"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654799    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d917584-197f-45cc-8586-4bdea2c623e1" containerName="revoke-keys"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654828    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654837    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654848    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" containerName="payload-processing"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654857    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" containerName="payload-processing"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654872    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6049be15-1e81-4fbe-a677-db9c2428eead" containerName="payload-pre-processing"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654880    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6049be15-1e81-4fbe-a677-db9c2428eead" containerName="payload-pre-processing"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654890    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" containerName="payload-processing"
Sep 10 21:55:46.654923 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654899    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" containerName="payload-processing"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654982    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d917584-197f-45cc-8586-4bdea2c623e1" containerName="revoke-keys"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.654996    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3332dbb5-87ef-4611-8e73-5416fcb6eea0" containerName="payload-processing"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.655007    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6049be15-1e81-4fbe-a677-db9c2428eead" containerName="payload-pre-processing"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.655015    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="24d9c529-cab8-48e4-b1ea-3c910fe6a579" containerName="payload-processing"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.655026    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="af9ca713-897d-4558-b42a-1087e06de374" containerName="istio-proxy"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.655036    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy"
Sep 10 21:55:46.655433 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.655045    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="35b3b44c-20fa-47dd-bffc-f03f1328242e" containerName="istio-proxy"
Sep 10 21:55:46.658385 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.658360    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.660595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.660569    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xhl4t\"/\"default-dockercfg-z8wtj\""
Sep 10 21:55:46.660595 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.660569    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xhl4t\"/\"kube-root-ca.crt\""
Sep 10 21:55:46.660761 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.660569    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xhl4t\"/\"openshift-service-ca.crt\""
Sep 10 21:55:46.667807 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.667759    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"]
Sep 10 21:55:46.846442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.846396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-lib-modules\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.846442 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.846443    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-proc\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.846659 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.846491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-sys\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.846659 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.846564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwn2l\" (UniqueName: \"kubernetes.io/projected/005d9389-e983-47f3-8cb0-51180ef44605-kube-api-access-nwn2l\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.846659 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.846598    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-podres\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-proc\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-sys\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947489 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwn2l\" (UniqueName: \"kubernetes.io/projected/005d9389-e983-47f3-8cb0-51180ef44605-kube-api-access-nwn2l\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947501    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-podres\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947530    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-proc\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-lib-modules\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-sys\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-podres\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.947727 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.947681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/005d9389-e983-47f3-8cb0-51180ef44605-lib-modules\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.955456 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.955433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwn2l\" (UniqueName: \"kubernetes.io/projected/005d9389-e983-47f3-8cb0-51180ef44605-kube-api-access-nwn2l\") pod \"perf-node-gather-daemonset-l2s6j\" (UID: \"005d9389-e983-47f3-8cb0-51180ef44605\") " pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:46.969458 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:46.969432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:47.238518 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:47.238433    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"]
Sep 10 21:55:47.242176 ip-10-0-128-173 kubenswrapper[2579]: W0910 21:55:47.242144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod005d9389_e983_47f3_8cb0_51180ef44605.slice/crio-69582b001249e0b96f26dc7e3a27e378096ddf5208149b8896a25f8a28f138d7 WatchSource:0}: Error finding container 69582b001249e0b96f26dc7e3a27e378096ddf5208149b8896a25f8a28f138d7: Status 404 returned error can't find the container with id 69582b001249e0b96f26dc7e3a27e378096ddf5208149b8896a25f8a28f138d7
Sep 10 21:55:47.914628 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:47.914589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j" event={"ID":"005d9389-e983-47f3-8cb0-51180ef44605","Type":"ContainerStarted","Data":"016799c65d389d907886fa8af90422555a65a8184f7ff0b29c14437f796e4c4b"}
Sep 10 21:55:47.915060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:47.914652    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j" event={"ID":"005d9389-e983-47f3-8cb0-51180ef44605","Type":"ContainerStarted","Data":"69582b001249e0b96f26dc7e3a27e378096ddf5208149b8896a25f8a28f138d7"}
Sep 10 21:55:47.915060 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:47.914722    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:47.933158 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:47.933113    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j" podStartSLOduration=1.933098762 podStartE2EDuration="1.933098762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 21:55:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 21:55:47.932245409 +0000 UTC m=+2184.427068949" watchObservedRunningTime="2026-09-10 21:55:47.933098762 +0000 UTC m=+2184.427922357"
Sep 10 21:55:48.387420 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:48.387391    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jc6p7_3ac182e9-57a2-4739-a9f2-d16d19831c1e/dns/0.log"
Sep 10 21:55:48.408537 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:48.408514    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jc6p7_3ac182e9-57a2-4739-a9f2-d16d19831c1e/kube-rbac-proxy/0.log"
Sep 10 21:55:48.510072 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:48.510046    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l69zp_763c18c2-136b-41a3-b05d-4907bf940f7b/dns-node-resolver/0.log"
Sep 10 21:55:49.126143 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:49.126111    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ns59t_a6388b71-4b19-4847-a45a-8c86d80b9bcd/node-ca/0.log"
Sep 10 21:55:50.169440 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:50.169398    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs_2842bf44-3f01-49e1-b186-c151e13b0778/istio-proxy/0.log"
Sep 10 21:55:50.392074 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:50.392044    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7q5k9_32c71e42-581a-4c01-9c95-12fc34c48b71/discovery/0.log"
Sep 10 21:55:51.480251 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:51.480223    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gcvtg_e0467875-9d2b-485c-9177-b9e075d8dce5/serve-healthcheck-canary/0.log"
Sep 10 21:55:52.106694 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:52.106667    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2g8rg_10b3bcec-62b2-40ab-a79a-cc6923ed2b61/kube-rbac-proxy/0.log"
Sep 10 21:55:52.131856 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:52.131831    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2g8rg_10b3bcec-62b2-40ab-a79a-cc6923ed2b61/exporter/0.log"
Sep 10 21:55:52.167383 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:52.167362    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2g8rg_10b3bcec-62b2-40ab-a79a-cc6923ed2b61/extractor/0.log"
Sep 10 21:55:53.931607 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:53.931503    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xhl4t/perf-node-gather-daemonset-l2s6j"
Sep 10 21:55:54.925900 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:54.925872    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vngxn_dd012855-932e-4c18-a81d-d2162ee5610f/server/0.log"
Sep 10 21:55:55.001836 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:55.001804    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dfmvp_06462169-a019-4ace-a50b-0ee178d060c4/manager/0.log"
Sep 10 21:55:55.162168 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:55.162141    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fc979fdc-sjwrs_5200c0c5-382f-4333-9890-325309b0cd2f/manager/0.log"
Sep 10 21:55:55.182154 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:55.182078    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56fc979fdc-sjwrs_5200c0c5-382f-4333-9890-325309b0cd2f/copy-manifests/0.log"
Sep 10 21:55:56.754340 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:56.754270    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5644d67b8d-dqfkx_dba3ebc2-c15b-444a-a228-97725058b983/manager/0.log"
Sep 10 21:55:57.553310 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:57.553282    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs_2842bf44-3f01-49e1-b186-c151e13b0778/istio-proxy/0.log"
Sep 10 21:55:57.766868 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:57.766837    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7q5k9_32c71e42-581a-4c01-9c95-12fc34c48b71/discovery/0.log"
Sep 10 21:55:59.482435 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:59.482402    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-zq2cs_2842bf44-3f01-49e1-b186-c151e13b0778/istio-proxy/0.log"
Sep 10 21:55:59.700182 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:55:59.700144    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7q5k9_32c71e42-581a-4c01-9c95-12fc34c48b71/discovery/0.log"
Sep 10 21:56:03.717991 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:03.717962    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6w4fx_4b8e000e-c4fb-4d2d-9bfe-2c9cba94ae24/kube-multus/0.log"
Sep 10 21:56:04.337935 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.337908    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/kube-multus-additional-cni-plugins/0.log"
Sep 10 21:56:04.366135 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.366111    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/egress-router-binary-copy/0.log"
Sep 10 21:56:04.397006 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.396987    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/cni-plugins/0.log"
Sep 10 21:56:04.417677 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.417652    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/bond-cni-plugin/0.log"
Sep 10 21:56:04.450199 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.450182    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/routeoverride-cni/0.log"
Sep 10 21:56:04.481644 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.481626    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/whereabouts-cni-bincopy/0.log"
Sep 10 21:56:04.502225 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.502208    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nkwbq_fb3a5920-8efd-4b31-9aa5-03ff83238bd9/whereabouts-cni/0.log"
Sep 10 21:56:04.692945 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.692869    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k2tdn_3f478cf5-0402-452a-8a3d-981ed11e30cf/network-metrics-daemon/0.log"
Sep 10 21:56:04.716944 ip-10-0-128-173 kubenswrapper[2579]: I0910 21:56:04.716926    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-k2tdn_3f478cf5-0402-452a-8a3d-981ed11e30cf/kube-rbac-proxy/0.log"