Sep 11 13:19:25.066077 ip-10-0-133-221 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:19:25.066097 ip-10-0-133-221 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:19:25.066110 ip-10-0-133-221 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:19:25.066399 ip-10-0-133-221 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:19:35.315264 ip-10-0-133-221 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 13:19:35.315280 ip-10-0-133-221 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 346c181e67074cd0a4fe97084f2384fc --
Sep 11 13:21:15.495821 ip-10-0-133-221 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:21:15.899030 ip-10-0-133-221 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:15.899030 ip-10-0-133-221 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:21:15.899030 ip-10-0-133-221 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:15.899030 ip-10-0-133-221 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:21:15.899030 ip-10-0-133-221 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:21:15.899980 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.899893    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:21:15.903149 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903133    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.903149 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903149    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903153    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903157    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903161    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903164    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903167    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903170    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903172    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903175    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903178    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903180    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903183    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903186    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903189    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903191    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903194    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903196    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903199    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903202    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903205    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.903212 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903207    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903210    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903213    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903216    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903219    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903222    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903225    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903228    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903230    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903233    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903236    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903239    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903242    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903244    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903247    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903249    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903252    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903255    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903257    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903260    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.903703 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903263    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903265    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903268    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903270    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903273    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903275    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903278    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903280    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903283    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903285    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903290    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903294    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903297    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903300    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903304    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903307    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903310    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903318    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903321    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.904181 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903324    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903327    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903329    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903333    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903335    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903338    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903341    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903344    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903346    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903349    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903352    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903355    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903358    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903361    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903363    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903366    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903370    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903374    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903377    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903381    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.904666 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903384    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903386    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903389    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903392    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903395    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903398    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903401    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903404    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903407    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903409    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903412    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903415    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903418    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903420    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903423    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903425    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903428    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903432    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.903435    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.905150 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904966    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904974    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904978    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904981    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904986    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904991    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904994    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904997    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.904999    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905002    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905005    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905007    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905010    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905012    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905015    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905018    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905021    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905023    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905027    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.905634 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905029    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905033    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905036    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905038    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905041    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905044    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905046    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905049    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905052    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905054    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905057    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905059    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905062    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905065    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905067    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905070    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905073    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905075    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905078    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.906144 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905087    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905089    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905092    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905095    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905097    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905100    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905102    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905105    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905107    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905110    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905112    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905115    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905119    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905122    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905126    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905128    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905131    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905134    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905136    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905139    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.906625 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905142    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905145    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905147    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905150    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905152    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905157    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905161    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905164    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905167    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905170    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905173    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905176    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905178    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905181    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905184    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905187    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905189    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905192    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905194    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.907120 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905197    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905200    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905204    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905207    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905210    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905213    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905215    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905218    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905221    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905224    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905226    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905229    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905232    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905234    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905237    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905240    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905243    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905245    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905248    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905251    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.907583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905253    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905256    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905667    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905673    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905676    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905679    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905683    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905685    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905688    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905691    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905693    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905696    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905698    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905701    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905704    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905706    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905709    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905712    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905715    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905719    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.908075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905722    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905724    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905727    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905730    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905732    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905735    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905740    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905743    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905746    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905749    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905752    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905754    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905757    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905759    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905762    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905765    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905767    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905770    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905773    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905776    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.908576 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905779    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905781    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905784    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905786    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905789    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905792    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905796    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905799    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905804    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905807    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905809    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905813    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905815    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905818    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905822    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905825    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905828    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905831    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905834    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.909064 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905836    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905839    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905841    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905844    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905846    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905849    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905852    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905854    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905857    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905860    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905862    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905865    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905867    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905871    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905874    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905876    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905879    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905882    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905884    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905887    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.909551 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905890    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905894    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905897    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905899    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905902    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905905    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905907    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905910    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905912    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905915    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905917    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905920    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905923    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905925    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905928    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905931    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905933    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905936    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905938    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905941    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905945    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.910030 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.905948    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906022    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906065    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906075    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906080    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906087    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906091    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906095    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906100    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906103    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906106    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906109    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906114    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906118    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906121    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906123    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906127    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906129    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906133    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906139    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906142    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906146    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906149    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906153    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:21:15.910550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906157    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906160    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906163    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906167    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906170    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906173    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906176    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906181    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906184    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906188    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906191    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906194    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906196    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906199    2576 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906208    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906214    2576 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906217    2576 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906220    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906224    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906227    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906236    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906239    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906242    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906245    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906248    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:21:15.911175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906251    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906254    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906257    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906260    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906263    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906266    2576 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906270    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906273    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906276    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906279    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906282    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906285    2576 flags.go:64] FLAG: --help="false"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906288    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906291    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906295    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906298    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906302    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906305    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906308    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906311    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906314    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906317    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906320    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906324    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:21:15.911810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906326    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906329    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906332    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906336    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906339    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906342    2576 flags.go:64] FLAG: --lock-file=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906345    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906348    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906351    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906357    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906360    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906362    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906365    2576 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906368    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906371    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906374    2576 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906377    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906382    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906385    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906389    2576 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906392    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906395    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906399    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906402    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906406    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:21:15.912427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906409    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906412    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906419    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906422    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906425    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906429    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906432    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906438    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906441    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906444    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906447    2576 flags.go:64] FLAG: --port="10250"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906452    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906455    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa96b643966a833b"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906458    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906461    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906464    2576 flags.go:64] FLAG: --register-node="true"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906467    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906471    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906474    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906476    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906479    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906483    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906486    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906489    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906504    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:21:15.913094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906507    2576 flags.go:64] FLAG: --runonce="false"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906510    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906514    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906517    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906521    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906524    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906527    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906531    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906534    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906537    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906540    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906543    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906546    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906549    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906553    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906555    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906561    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906564    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906568    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906573    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906576    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906579    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906582    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906585    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906588    2576 flags.go:64] FLAG: --v="2"
Sep 11 13:21:15.913856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906592    2576 flags.go:64] FLAG: --version="false"
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906596    2576 flags.go:64] FLAG: --vmodule=""
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906600    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.906603    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906733    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906737    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906741    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906744    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906747    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906750    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906753    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906757    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906760    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906763    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906766    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906769    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906771    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906774    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906777    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906780    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906782    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.914514 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906785    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906794    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906796    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906799    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906802    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906805    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906808    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906810    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906813    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906816    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906818    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906821    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906824    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906828    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906832    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906835    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906838    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906841    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906843    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.915110 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906846    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906849    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906852    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906855    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906858    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906860    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906863    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906865    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906868    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906870    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906873    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906875    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906878    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906881    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906883    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906894    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906897    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906900    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906903    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906905    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.915591 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906908    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906911    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906913    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906916    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906919    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906921    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906924    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906927    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906929    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906932    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906934    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906937    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906939    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906943    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906946    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906950    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906954    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906956    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906960    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.916082 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906963    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906966    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906969    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906971    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906974    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906976    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906979    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906982    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906984    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906993    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906996    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.906998    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907007    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907009    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907012    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907015    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907018    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907020    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907023    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907025    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.916567 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907028    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907031    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907033    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.907036    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.907888    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.915866    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.915882    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.915901    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.915907    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.915981    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.915986    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.915989    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.915993    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.915996    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.917075 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916000    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916003    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916006    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916009    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916012    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916014    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916017    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916020    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916023    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916025    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916028    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916031    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916033    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916036    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916039    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916041    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916044    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916047    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916050    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916052    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.917656 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916055    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916057    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916060    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916062    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916067    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916070    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916073    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916076    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916079    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916081    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916085    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916087    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916090    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916093    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916096    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916099    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916101    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916104    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916108    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916112    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.918406 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916115    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916118    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916121    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916124    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916127    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916130    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916133    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916136    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916138    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916141    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916144    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916146    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916149    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916152    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916154    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916157    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916161    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916165    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916169    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916172    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.918990 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916176    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916179    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916182    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916185    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916188    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916191    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916194    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916197    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916201    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916205    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916208    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916211    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916213    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916216    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916220    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916222    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916225    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916228    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916231    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.919481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916233    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916236    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916239    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916242    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916244    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916247    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916249    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916252    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916255    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916258    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916261    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916264    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916267    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916270    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916273    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.919971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.916278    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916422    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916427    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916430    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916434    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916437    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916440    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916443    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916446    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916461    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916466    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916471    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916475    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916478    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916481    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916484    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916487    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916490    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916510    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:21:15.920339 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916513    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916516    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916519    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916523    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916526    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916529    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916532    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916535    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916539    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916542    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916546    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916549    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916551    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916554    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916557    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916561    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916565    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916568    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916571    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:21:15.920813 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916573    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916576    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916579    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916582    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916584    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916587    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916591    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916595    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916598    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916601    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916604    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916607    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916609    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916612    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916615    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916618    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916620    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916623    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916625    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916628    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:21:15.921278 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916631    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916634    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916637    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916640    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916643    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916646    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916648    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916651    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916654    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916657    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916659    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916662    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916665    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916667    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916670    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916672    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916675    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916678    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916681    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916684    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:21:15.921836 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916686    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916689    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916692    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916694    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916697    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916700    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916703    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916706    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916709    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916711    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916714    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916717    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916719    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916722    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916725    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916728    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916731    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916734    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916736    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916739    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:21:15.922327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916742    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:15.916744    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.916749    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.916862    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.920003    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.921051    2576 server.go:1019] "Starting client certificate rotation"
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.921146    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:21:15.922829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.921984    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:21:15.923958 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.923941    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:21:15.927125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.927094    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:21:15.939857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.939835    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:21:15.943954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.943938    2576 log.go:25] "Validated CRI v1 image API"
Sep 11 13:21:15.945600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.945585    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:21:15.951611 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.951587    2576 fs.go:135] Filesystem UUIDs: map[18a14e7a-170c-4c87-b1ee-29c9d6fa8078:/dev/nvme0n1p4 1d4893f0-ef52-463f-ae39-93bccddd50b4:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 11 13:21:15.951669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.951611    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:21:15.957259 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.957144    2576 manager.go:216] Machine: {Timestamp:2026-09-11 13:21:15.955799216 +0000 UTC m=+0.349486621 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093323 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d14c627dfbc5050de06616d3d70a0 SystemUUID:ec2d14c6-27df-bc50-50de-06616d3d70a0 BootID:346c181e-6707-4cd0-a4fe-97084f2384fc Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3e:f4:f7:c7:ad Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3e:f4:f7:c7:ad Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:1d:db:dd:b2:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:21:15.957259 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.957258    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:21:15.957375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.957343    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:21:15.961030 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.961013    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:21:15.961097 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.961056    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:21:15.961241 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.961086    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-221.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:21:15.962022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.962012    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:21:15.962055 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.962026    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:21:15.962055 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.962033    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:21:15.962943 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.962933    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:21:15.964045 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.964036    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:21:15.967998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.967982    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:21:15.968123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968113    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:21:15.968161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968146    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:21:15.968161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968156    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:21:15.968237 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968169    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:21:15.968237 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968177    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:21:15.968237 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968186    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:21:15.968612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968600    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:21:15.968645 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.968619    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:21:15.970943 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.970925    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:21:15.973718 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.973704    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:21:15.973774 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.973734    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:21:15.975803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975785    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:21:15.975803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975802    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975809    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975814    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975823    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975828    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975841    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975846    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975852    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975857    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975866    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975871    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975894    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:21:15.975932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.975899    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:21:15.979789 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.979776    2576 server.go:1268] "Started kubelet"
Sep 11 13:21:15.979980 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.979921    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:21:15.980087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.980003    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:21:15.980686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.980647    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:21:15.980811 ip-10-0-133-221 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:21:15.981929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.981663    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:21:15.984712 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.984158    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:21:15.985300 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.985283    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:21:15.990631 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.990615    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:21:15.990717 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.990689    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:15.990717 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.990696    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:21:15.991254 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.991225    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:21:15.991409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991376    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:21:15.991409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991378    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:21:15.991540 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991416    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:21:15.991597 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991548    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:21:15.991597 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991558    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:21:15.991695 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991624    2576 factory.go:55] Registering systemd factory
Sep 11 13:21:15.991695 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991641    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:21:15.991794 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.991713    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:21:15.991922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991907    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-221.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:21:15.991996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991984    2576 factory.go:153] Registering CRI-O factory
Sep 11 13:21:15.992043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.991999    2576 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:21:15.992089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.992048    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:21:15.992089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.992064    2576 factory.go:103] Registering Raw factory
Sep 11 13:21:15.992089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.992082    2576 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:21:15.992380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:15.992368    2576 manager.go:349] Starting recovery of all containers
Sep 11 13:21:15.992849 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.991699    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df18bee5aa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:15.979744682 +0000 UTC m=+0.373432084,LastTimestamp:2026-09-11 13:21:15.979744682 +0000 UTC m=+0.373432084,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:15.996237 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.996197    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-221.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:21:15.996584 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:15.996542    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:21:16.006006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.005991    2576 manager.go:354] Recovery completed
Sep 11 13:21:16.011130 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.011012    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.013074 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013056    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.013144 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013087    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.013144 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013098    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.013724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013709    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:21:16.013724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013717    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.013724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013723    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:21:16.013872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.013741    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:21:16.014406 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.014390    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.014470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.014416    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.014470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.014426    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.017423 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.017359    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abb7454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.013073492 +0000 UTC m=+0.406760897,LastTimestamp:2026-09-11 13:21:16.013073492 +0000 UTC m=+0.406760897,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.026681 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.026659    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4h6rs"
Sep 11 13:21:16.031923 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.031820    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abbbd22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.01309213 +0000 UTC m=+0.406779532,LastTimestamp:2026-09-11 13:21:16.01309213 +0000 UTC m=+0.406779532,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.036691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.036670    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4h6rs"
Sep 11 13:21:16.043253 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.043182    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abbe33b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.013101883 +0000 UTC m=+0.406789285,LastTimestamp:2026-09-11 13:21:16.013101883 +0000 UTC m=+0.406789285,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.058250    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-221.ec2.internal.18d446df1abb7454\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abb7454  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.013073492 +0000 UTC m=+0.406760897,LastTimestamp:2026-09-11 13:21:16.014404146 +0000 UTC m=+0.408091551,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.071462    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-221.ec2.internal.18d446df1abbbd22\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abbbd22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.01309213 +0000 UTC m=+0.406779532,LastTimestamp:2026-09-11 13:21:16.014420667 +0000 UTC m=+0.408108068,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.085347    2576 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-221.ec2.internal.18d446df1abbe33b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-221.ec2.internal.18d446df1abbe33b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-221.ec2.internal,UID:ip-10-0-133-221.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-221.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-221.ec2.internal,},FirstTimestamp:2026-09-11 13:21:16.013101883 +0000 UTC m=+0.406789285,LastTimestamp:2026-09-11 13:21:16.014429901 +0000 UTC m=+0.408117303,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-221.ec2.internal,}"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.091453    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.154109    2576 policy_none.go:49] "None policy: Start"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.154159    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:21:16.154697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.154175    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:21:16.155333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.155312    2576 policy_none.go:47] "Start"
Sep 11 13:21:16.184048 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184029    2576 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:21:16.184150 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.184084    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:21:16.184150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184098    2576 server.go:87] "Starting device plugin registration server"
Sep 11 13:21:16.184464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184449    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:21:16.184561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184464    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:21:16.184612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184576    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:21:16.184704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184681    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:21:16.184704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.184693    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:21:16.185367 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.185352    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:21:16.185443 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.185393    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.211974 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.211944    2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-221.ec2.internal\" not found" node="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.240545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.240510    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:21:16.241762 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.241745    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:21:16.241847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.241797    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:21:16.241847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.241822    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:21:16.241937 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.241927    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:21:16.247835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.247818    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:16.285413 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.285389    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.286378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.286361    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.286470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.286396    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.286470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.286413    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.286470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.286446    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.307861 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.307836    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.307861 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.307861    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-221.ec2.internal\": node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.343051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.343022    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"]
Sep 11 13:21:16.343162 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.343111    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.344074 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.344058    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.344167 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.344086    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.344167 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.344096    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.344678 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.344664    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.345645 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.345631    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.345756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.345739    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.345822 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.345786    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.346307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346290    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.346440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346319    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.346440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346336    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.346440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346370    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.346440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346396    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.346440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.346407    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.347800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.347787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.347846 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.347810    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:16.348547 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.348530    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:16.348626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.348560    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:16.348626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.348576    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:16.377233 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.377215    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-221.ec2.internal\" not found" node="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.381997 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.381981    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-221.ec2.internal\" not found" node="ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.394935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.394916    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.395017 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.394940    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e834e13edb4b1c98f5d9c9a73bedc2fb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-221.ec2.internal\" (UID: \"e834e13edb4b1c98f5d9c9a73bedc2fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.395017 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.394956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.444994 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.444919    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.495878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e834e13edb4b1c98f5d9c9a73bedc2fb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-221.ec2.internal\" (UID: \"e834e13edb4b1c98f5d9c9a73bedc2fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.495878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.496093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.496093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e834e13edb4b1c98f5d9c9a73bedc2fb-config\") pod \"kube-apiserver-proxy-ip-10-0-133-221.ec2.internal\" (UID: \"e834e13edb4b1c98f5d9c9a73bedc2fb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.496093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.496093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.495946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c415eb3b64e424c5a88aa6d0ac326c37-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal\" (UID: \"c415eb3b64e424c5a88aa6d0ac326c37\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.545780 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.545755    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.646831 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.646788    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.680323 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.680294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.685090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.685047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:16.747981 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.747899    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.848716 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.848678    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:16.921255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.921226    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:21:16.921853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:16.921377    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:21:16.950747 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:16.950718    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.001421 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.001349    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:17.044636 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.044605    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:21:17.045871 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.045848    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:21:17.046010 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.045882    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:21:17.046010 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.045897    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:21:17.051188 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.051169    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.152176 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.152137    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.157672 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.157635    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:16:16 +0000 UTC" deadline="2028-04-29 06:27:48.810212891 +0000 UTC"
Sep 11 13:21:17.157672 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.157669    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14297h6m31.652546835s"
Sep 11 13:21:17.253116 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.253037    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.255811 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.255794    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:17.353745 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.353703    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.454439 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.454398    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-221.ec2.internal\" not found"
Sep 11 13:21:17.485934 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:17.485903    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc415eb3b64e424c5a88aa6d0ac326c37.slice/crio-c0a80c9295487dbbe99626b69e5f6b0603015f00c59e50b0dee1f134c2260ae5 WatchSource:0}: Error finding container c0a80c9295487dbbe99626b69e5f6b0603015f00c59e50b0dee1f134c2260ae5: Status 404 returned error can't find the container with id c0a80c9295487dbbe99626b69e5f6b0603015f00c59e50b0dee1f134c2260ae5
Sep 11 13:21:17.486344 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:17.486320    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode834e13edb4b1c98f5d9c9a73bedc2fb.slice/crio-520c01d8f997b8ec9da9fab5dcfbc3d24f5a7c2b4002370ae2d955f04fd58f83 WatchSource:0}: Error finding container 520c01d8f997b8ec9da9fab5dcfbc3d24f5a7c2b4002370ae2d955f04fd58f83: Status 404 returned error can't find the container with id 520c01d8f997b8ec9da9fab5dcfbc3d24f5a7c2b4002370ae2d955f04fd58f83
Sep 11 13:21:17.491787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.491766    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:21:17.496355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.496335    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:17.591996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.591890    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:17.610236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.610196    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:21:17.611193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.611177    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal"
Sep 11 13:21:17.623767 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.623748    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:21:17.884399 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.884311    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:21:17.969277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.969253    2576 apiserver.go:52] "Watching apiserver"
Sep 11 13:21:17.976732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.976713    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:21:17.978547 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.978522    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-crjpf","kube-system/konnectivity-agent-nvmtf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5","openshift-cluster-node-tuning-operator/tuned-qbc8w","openshift-image-registry/node-ca-tq76r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal","openshift-network-diagnostics/network-check-target-vdlnp","openshift-network-operator/iptables-alerter-qqjvw","kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal","openshift-dns/node-resolver-tkfh5","openshift-multus/multus-additional-cni-plugins-ck5bd","openshift-multus/multus-f7lfj","openshift-multus/network-metrics-daemon-rgwvv","openshift-ovn-kubernetes/ovnkube-node-wz6r9"]
Sep 11 13:21:17.981042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.981023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:17.981130 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.981097    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:17.983340 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.983317    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:17.983433 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:17.983400    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:17.986077 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.986053    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:17.988584 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.988554    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:17.991334 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.991312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:17.992285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.991750    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:21:17.993682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.993659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:17.995594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.995102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:21:17.995594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.995179    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:21:17.995950 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.995821    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:21:17.995950 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.995853    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bmpx7\""
Sep 11 13:21:17.996616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.996212    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:21:17.996616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.996247    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:21:17.996616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.996458    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:21:17.997037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.997011    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:21:17.997559 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.997320    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:21:17.997762 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.997745    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9q8ns\""
Sep 11 13:21:17.997934 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.997911    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-58zbx\""
Sep 11 13:21:17.998101 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.998087    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:17.999333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.999312    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:21:17.999434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:17.999387    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:21:18.000629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.000563    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:21:18.000736 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.000725    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:21:18.000977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.000959    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.001074 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.001035    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:18.002958 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.002936    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:21:18.003117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.003093    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:21:18.003204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.003186    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:21:18.003462 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.003279    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gsww4\""
Sep 11 13:21:18.004285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.003698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.005735 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005715    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:21:18.005831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.005831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005778    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-modprobe-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.005831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-systemd\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.005997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-var-lib-kubelet\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.005997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-netns\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.005997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-node-log\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.005997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.005986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-log-socket\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006019    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-env-overrides\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-socket-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-sys\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93b4f034-59d0-442e-b948-370d656307d9-host\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-lib-modules\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-etc-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-device-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysconfig\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-tmp\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006378    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd8dt\" (UniqueName: \"kubernetes.io/projected/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-kube-api-access-jd8dt\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-systemd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-var-lib-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-script-lib\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zptg\" (UniqueName: \"kubernetes.io/projected/494d8db9-4794-4883-a7f4-08f69f46dd19-kube-api-access-7zptg\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.006563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-registration-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006607    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nqqh\" (UniqueName: \"kubernetes.io/projected/b2e18965-023d-4944-acfc-dc7878dcf4f5-kube-api-access-5nqqh\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-kubernetes\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqr75\" (UniqueName: \"kubernetes.io/projected/93b4f034-59d0-442e-b948-370d656307d9-kube-api-access-zqr75\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-sys-fs\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-conf\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93b4f034-59d0-442e-b948-370d656307d9-serviceca\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006824    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-kubelet\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006848    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/494d8db9-4794-4883-a7f4-08f69f46dd19-ovn-node-metrics-cert\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.006981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.007036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-run\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-tuned\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-systemd-units\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-slash\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwktz\" (UniqueName: \"kubernetes.io/projected/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-kube-api-access-nwktz\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007187    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-host\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-ovn\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-bin\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-netd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-config\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007472    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zdc6h\""
Sep 11 13:21:18.007746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.007477    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:21:18.009454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.009428    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.010046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.010022    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:21:18.010361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.010342    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:21:18.010557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.010381    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q5hwt\""
Sep 11 13:21:18.011221 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.011193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x4j72\""
Sep 11 13:21:18.012363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.012342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.012986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.012965    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:21:18.013079 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013065    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:21:18.013255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:21:18.013663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013241    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:21:18.013663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013632    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2crdk\""
Sep 11 13:21:18.013841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013665    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:21:18.013841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013700    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:21:18.013841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.013640    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:21:18.017911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.017890    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:21:18.018431 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.018410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ptcgg\""
Sep 11 13:21:18.021628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.021376    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:21:18.041420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.041394    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4dcpj"
Sep 11 13:21:18.052171 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.052145    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4dcpj"
Sep 11 13:21:18.093893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.093862    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:21:18.107603 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.107764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-modprobe-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-var-lib-kubelet\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-netns\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.107764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107711    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-etc-selinux\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-node-log\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-node-log\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-cni-binary-copy\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-modprobe-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-lib-modules\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107934    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-var-lib-kubelet\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-etc-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.107986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.107983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-kubelet\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108062    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-host-slash\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108072    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-d\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec4d4872-4d83-4d4a-a78f-6945f2447a76-hosts-file\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-etc-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108115    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-tmp\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108122    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-netns\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-var-lib-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zptg\" (UniqueName: \"kubernetes.io/projected/494d8db9-4794-4883-a7f4-08f69f46dd19-kube-api-access-7zptg\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-var-lib-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-hostroot\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4d4872-4d83-4d4a-a78f-6945f2447a76-tmp-dir\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdz2q\" (UniqueName: \"kubernetes.io/projected/ec4d4872-4d83-4d4a-a78f-6945f2447a76-kube-api-access-hdz2q\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.108454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nqqh\" (UniqueName: \"kubernetes.io/projected/b2e18965-023d-4944-acfc-dc7878dcf4f5-kube-api-access-5nqqh\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108450    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-kubernetes\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108489    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqr75\" (UniqueName: \"kubernetes.io/projected/93b4f034-59d0-442e-b948-370d656307d9-kube-api-access-zqr75\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-conf-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phg7l\" (UniqueName: \"kubernetes.io/projected/d883a3c1-c563-425c-a585-b04894de1190-kube-api-access-phg7l\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-lib-modules\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-kubelet-config\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-sys-fs\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108609    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-conf\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93b4f034-59d0-442e-b948-370d656307d9-serviceca\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.108712    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njzrb\" (UniqueName: \"kubernetes.io/projected/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-kube-api-access-njzrb\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.108788    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:18.608766157 +0000 UTC m=+3.002453550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:18.109288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-tuned\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108815    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-ovn\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cnibin\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-sys-fs\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108891    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-etc-kubernetes\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-ovn\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-run-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysctl-conf\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.108633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-kubernetes\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109196    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lwt7\" (UniqueName: \"kubernetes.io/projected/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-kube-api-access-4lwt7\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-k8s-cni-cncf-io\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-systemd\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-systemd\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-log-socket\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-env-overrides\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fab8561-db61-40c9-9972-73e8e638cc7f-agent-certs\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109479    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-log-socket\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93b4f034-59d0-442e-b948-370d656307d9-serviceca\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-multus\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-socket-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-sys\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-socket-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-sys\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93b4f034-59d0-442e-b948-370d656307d9-host\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93b4f034-59d0-442e-b948-370d656307d9-host\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-socket-dir-parent\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-device-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysconfig\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd8dt\" (UniqueName: \"kubernetes.io/projected/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-kube-api-access-jd8dt\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-systemd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.110872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-device-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-script-lib\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-system-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-registration-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-env-overrides\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-systemd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.109933    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-system-cni-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-bin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-sysconfig\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110029    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-multus-daemon-config\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-kubelet\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110126    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/494d8db9-4794-4883-a7f4-08f69f46dd19-ovn-node-metrics-cert\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b2e18965-023d-4944-acfc-dc7878dcf4f5-registration-dir\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-kubelet\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110220    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fab8561-db61-40c9-9972-73e8e638cc7f-konnectivity-ca\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-multus-certs\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.111709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-dbus\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-run\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110416    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-systemd-units\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-slash\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-run\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-bin\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-run-openvswitch\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-slash\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-script-lib\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-systemd-units\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-bin\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-config\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-netns\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110658    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-iptables-alerter-script\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwktz\" (UniqueName: \"kubernetes.io/projected/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-kube-api-access-nwktz\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110730    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-host\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-netd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.112609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/494d8db9-4794-4883-a7f4-08f69f46dd19-host-cni-netd\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.113453 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-host\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.113453 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.110934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-cnibin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.113453 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.111017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/494d8db9-4794-4883-a7f4-08f69f46dd19-ovnkube-config\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.113940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.113921    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-tmp\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.114066 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.114043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-etc-tuned\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.115676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.115655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/494d8db9-4794-4883-a7f4-08f69f46dd19-ovn-node-metrics-cert\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.129410 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.129358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zptg\" (UniqueName: \"kubernetes.io/projected/494d8db9-4794-4883-a7f4-08f69f46dd19-kube-api-access-7zptg\") pod \"ovnkube-node-wz6r9\" (UID: \"494d8db9-4794-4883-a7f4-08f69f46dd19\") " pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.140667 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.140607    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:18.140667 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.140640    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:18.140667 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.140659    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:18.140916 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.140771    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:18.640749512 +0000 UTC m=+3.034436928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:18.142889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.142842    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nqqh\" (UniqueName: \"kubernetes.io/projected/b2e18965-023d-4944-acfc-dc7878dcf4f5-kube-api-access-5nqqh\") pod \"aws-ebs-csi-driver-node-sp7s5\" (UID: \"b2e18965-023d-4944-acfc-dc7878dcf4f5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.143319 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.143292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqr75\" (UniqueName: \"kubernetes.io/projected/93b4f034-59d0-442e-b948-370d656307d9-kube-api-access-zqr75\") pod \"node-ca-tq76r\" (UID: \"93b4f034-59d0-442e-b948-370d656307d9\") " pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.144331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.144306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwktz\" (UniqueName: \"kubernetes.io/projected/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-kube-api-access-nwktz\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.144331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.144324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd8dt\" (UniqueName: \"kubernetes.io/projected/a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8-kube-api-access-jd8dt\") pod \"tuned-qbc8w\" (UID: \"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8\") " pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.211908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.211872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.211908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.211915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-kubelet\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.211939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-host-slash\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.211962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec4d4872-4d83-4d4a-a78f-6945f2447a76-hosts-file\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.211989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-hostroot\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4d4872-4d83-4d4a-a78f-6945f2447a76-tmp-dir\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdz2q\" (UniqueName: \"kubernetes.io/projected/ec4d4872-4d83-4d4a-a78f-6945f2447a76-kube-api-access-hdz2q\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.212164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212176    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ec4d4872-4d83-4d4a-a78f-6945f2447a76-hosts-file\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-conf-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phg7l\" (UniqueName: \"kubernetes.io/projected/d883a3c1-c563-425c-a585-b04894de1190-kube-api-access-phg7l\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212231    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-hostroot\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-kubelet\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-host-slash\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-conf-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-kubelet-config\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njzrb\" (UniqueName: \"kubernetes.io/projected/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-kube-api-access-njzrb\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cnibin\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-etc-kubernetes\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lwt7\" (UniqueName: \"kubernetes.io/projected/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-kube-api-access-4lwt7\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec4d4872-4d83-4d4a-a78f-6945f2447a76-tmp-dir\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-k8s-cni-cncf-io\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.212514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fab8561-db61-40c9-9972-73e8e638cc7f-agent-certs\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-multus\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-socket-dir-parent\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-system-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-kubelet-config\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-system-cni-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-k8s-cni-cncf-io\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.212571    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-etc-kubernetes\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-bin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.212737    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:18.712716512 +0000 UTC m=+3.106403923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cnibin\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-multus-daemon-config\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-bin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-system-cni-dir\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fab8561-db61-40c9-9972-73e8e638cc7f-konnectivity-ca\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.213387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-system-cni-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-multus-certs\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-multus-socket-dir-parent\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-dbus\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-netns\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-iptables-alerter-script\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-cnibin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.212993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/45c1f894-db5d-445d-a062-41f989520fa2-dbus\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-cni-binary-copy\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213116    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-var-lib-cni-multus\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-netns\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-host-run-multus-certs\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213289    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d883a3c1-c563-425c-a585-b04894de1190-cnibin\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3fab8561-db61-40c9-9972-73e8e638cc7f-konnectivity-ca\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.214280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-multus-daemon-config\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.215202 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d883a3c1-c563-425c-a585-b04894de1190-cni-binary-copy\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.215202 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213542    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.215202 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.213823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-iptables-alerter-script\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.215887 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.215847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3fab8561-db61-40c9-9972-73e8e638cc7f-agent-certs\") pod \"konnectivity-agent-nvmtf\" (UID: \"3fab8561-db61-40c9-9972-73e8e638cc7f\") " pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.230049 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.230021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phg7l\" (UniqueName: \"kubernetes.io/projected/d883a3c1-c563-425c-a585-b04894de1190-kube-api-access-phg7l\") pod \"multus-f7lfj\" (UID: \"d883a3c1-c563-425c-a585-b04894de1190\") " pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.233316 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.233269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njzrb\" (UniqueName: \"kubernetes.io/projected/a2949ce6-3bb6-4d10-b03c-0fb47be3bbea-kube-api-access-njzrb\") pod \"multus-additional-cni-plugins-ck5bd\" (UID: \"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea\") " pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.237280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.237257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdz2q\" (UniqueName: \"kubernetes.io/projected/ec4d4872-4d83-4d4a-a78f-6945f2447a76-kube-api-access-hdz2q\") pod \"node-resolver-tkfh5\" (UID: \"ec4d4872-4d83-4d4a-a78f-6945f2447a76\") " pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.241059 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.241036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lwt7\" (UniqueName: \"kubernetes.io/projected/17d15b5c-ceaa-4092-96ad-c48f5ae808cd-kube-api-access-4lwt7\") pod \"iptables-alerter-qqjvw\" (UID: \"17d15b5c-ceaa-4092-96ad-c48f5ae808cd\") " pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.251069 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.249106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal" event={"ID":"e834e13edb4b1c98f5d9c9a73bedc2fb","Type":"ContainerStarted","Data":"520c01d8f997b8ec9da9fab5dcfbc3d24f5a7c2b4002370ae2d955f04fd58f83"}
Sep 11 13:21:18.251192 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.251174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal" event={"ID":"c415eb3b64e424c5a88aa6d0ac326c37","Type":"ContainerStarted","Data":"c0a80c9295487dbbe99626b69e5f6b0603015f00c59e50b0dee1f134c2260ae5"}
Sep 11 13:21:18.305177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.305141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5"
Sep 11 13:21:18.314426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.314268    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbc8w"
Sep 11 13:21:18.318320 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.318236    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e18965_023d_4944_acfc_dc7878dcf4f5.slice/crio-1ff8016a4cf97cd3552adc179ac7516228db6286dadd8d613199cb11e79ecebf WatchSource:0}: Error finding container 1ff8016a4cf97cd3552adc179ac7516228db6286dadd8d613199cb11e79ecebf: Status 404 returned error can't find the container with id 1ff8016a4cf97cd3552adc179ac7516228db6286dadd8d613199cb11e79ecebf
Sep 11 13:21:18.325701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.325675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tq76r"
Sep 11 13:21:18.326537 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.326487    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ecaf2b_b9f6_4edf_8cf9_e561b48b56d8.slice/crio-ecabae442bc2494053e48d7cb8d685c7c4e8179f239aeaae9e02d7b569dcd2ac WatchSource:0}: Error finding container ecabae442bc2494053e48d7cb8d685c7c4e8179f239aeaae9e02d7b569dcd2ac: Status 404 returned error can't find the container with id ecabae442bc2494053e48d7cb8d685c7c4e8179f239aeaae9e02d7b569dcd2ac
Sep 11 13:21:18.333481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.333293    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:18.340588 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.340553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b4f034_59d0_442e_b948_370d656307d9.slice/crio-0514b8ae3488cfc92b7fefea070fcd0b0d12933cdde01bb278cbbc675352c233 WatchSource:0}: Error finding container 0514b8ae3488cfc92b7fefea070fcd0b0d12933cdde01bb278cbbc675352c233: Status 404 returned error can't find the container with id 0514b8ae3488cfc92b7fefea070fcd0b0d12933cdde01bb278cbbc675352c233
Sep 11 13:21:18.342357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.342203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qqjvw"
Sep 11 13:21:18.344837 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.344775    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494d8db9_4794_4883_a7f4_08f69f46dd19.slice/crio-c423dfa22bba1c57a3058e8b2abcf6b3ab8b7896a38411189ce3a31c1dcee8bb WatchSource:0}: Error finding container c423dfa22bba1c57a3058e8b2abcf6b3ab8b7896a38411189ce3a31c1dcee8bb: Status 404 returned error can't find the container with id c423dfa22bba1c57a3058e8b2abcf6b3ab8b7896a38411189ce3a31c1dcee8bb
Sep 11 13:21:18.351154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.351116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:18.354537 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.354484    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d15b5c_ceaa_4092_96ad_c48f5ae808cd.slice/crio-a579aae5927573b2b1819db5cb1134d48bfd84fd5018c16f512ac1860db770d9 WatchSource:0}: Error finding container a579aae5927573b2b1819db5cb1134d48bfd84fd5018c16f512ac1860db770d9: Status 404 returned error can't find the container with id a579aae5927573b2b1819db5cb1134d48bfd84fd5018c16f512ac1860db770d9
Sep 11 13:21:18.359041 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.358849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tkfh5"
Sep 11 13:21:18.362780 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.362745    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fab8561_db61_40c9_9972_73e8e638cc7f.slice/crio-98ce3f4514793fd66d15c8e1ac5b3d2a4d64ef1e25d0146e7aef6b4b7808039d WatchSource:0}: Error finding container 98ce3f4514793fd66d15c8e1ac5b3d2a4d64ef1e25d0146e7aef6b4b7808039d: Status 404 returned error can't find the container with id 98ce3f4514793fd66d15c8e1ac5b3d2a4d64ef1e25d0146e7aef6b4b7808039d
Sep 11 13:21:18.369468 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.368403    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ck5bd"
Sep 11 13:21:18.375232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.375207    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f7lfj"
Sep 11 13:21:18.383661 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.383628    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2949ce6_3bb6_4d10_b03c_0fb47be3bbea.slice/crio-e7b629ea47918d07de7e056948647ced1efea8b5c71907343c7d4b34f1604904 WatchSource:0}: Error finding container e7b629ea47918d07de7e056948647ced1efea8b5c71907343c7d4b34f1604904: Status 404 returned error can't find the container with id e7b629ea47918d07de7e056948647ced1efea8b5c71907343c7d4b34f1604904
Sep 11 13:21:18.388194 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:18.388166    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd883a3c1_c563_425c_a585_b04894de1190.slice/crio-636e2fb16cc342ea8a0dd459ab98e81fbdc2d2deb12194ff71cfa4759c70914e WatchSource:0}: Error finding container 636e2fb16cc342ea8a0dd459ab98e81fbdc2d2deb12194ff71cfa4759c70914e: Status 404 returned error can't find the container with id 636e2fb16cc342ea8a0dd459ab98e81fbdc2d2deb12194ff71cfa4759c70914e
Sep 11 13:21:18.616378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.616335    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:18.616596 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.616474    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:18.616596 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.616570    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:19.616527545 +0000 UTC m=+4.010214955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:18.716859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.716767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:18.717027 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.716947    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:18.717027 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.716985    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:18.717027 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.716997    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:18.717201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:18.716947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:18.717201 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.717043    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:18.717201 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.717050    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:19.717033657 +0000 UTC m=+4.110721048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:18.717201 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:18.717132    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:19.717113876 +0000 UTC m=+4.110801267 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:19.056271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.056083    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:16:18 +0000 UTC" deadline="2028-06-18 07:10:43.295452248 +0000 UTC"
Sep 11 13:21:19.056271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.056159    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15497h49m24.239297552s"
Sep 11 13:21:19.242125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.242082    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:19.242316 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.242101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:19.242316 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.242214    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:19.242316 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.242263    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:19.256477 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.256407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tq76r" event={"ID":"93b4f034-59d0-442e-b948-370d656307d9","Type":"ContainerStarted","Data":"0514b8ae3488cfc92b7fefea070fcd0b0d12933cdde01bb278cbbc675352c233"}
Sep 11 13:21:19.258367 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.258300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbc8w" event={"ID":"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8","Type":"ContainerStarted","Data":"ecabae442bc2494053e48d7cb8d685c7c4e8179f239aeaae9e02d7b569dcd2ac"}
Sep 11 13:21:19.260337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.260265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerStarted","Data":"e7b629ea47918d07de7e056948647ced1efea8b5c71907343c7d4b34f1604904"}
Sep 11 13:21:19.262253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.262183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tkfh5" event={"ID":"ec4d4872-4d83-4d4a-a78f-6945f2447a76","Type":"ContainerStarted","Data":"740b0e2b5edc1f968a5e7498c2c0ba528acff26ada4182bd2db1319e568a91f8"}
Sep 11 13:21:19.265222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.265163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nvmtf" event={"ID":"3fab8561-db61-40c9-9972-73e8e638cc7f","Type":"ContainerStarted","Data":"98ce3f4514793fd66d15c8e1ac5b3d2a4d64ef1e25d0146e7aef6b4b7808039d"}
Sep 11 13:21:19.268206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.268180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqjvw" event={"ID":"17d15b5c-ceaa-4092-96ad-c48f5ae808cd","Type":"ContainerStarted","Data":"a579aae5927573b2b1819db5cb1134d48bfd84fd5018c16f512ac1860db770d9"}
Sep 11 13:21:19.270051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.269980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5" event={"ID":"b2e18965-023d-4944-acfc-dc7878dcf4f5","Type":"ContainerStarted","Data":"1ff8016a4cf97cd3552adc179ac7516228db6286dadd8d613199cb11e79ecebf"}
Sep 11 13:21:19.272007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.271959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7lfj" event={"ID":"d883a3c1-c563-425c-a585-b04894de1190","Type":"ContainerStarted","Data":"636e2fb16cc342ea8a0dd459ab98e81fbdc2d2deb12194ff71cfa4759c70914e"}
Sep 11 13:21:19.273863 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.273771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"c423dfa22bba1c57a3058e8b2abcf6b3ab8b7896a38411189ce3a31c1dcee8bb"}
Sep 11 13:21:19.626114 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.625847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:19.626337 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.626140    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:19.626337 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.626209    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:21.626193034 +0000 UTC m=+6.019880447 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:19.726986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.726946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:19.727195 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:19.727022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:19.727195 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727184    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:19.727314 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727260    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:21.727231834 +0000 UTC m=+6.120919253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:19.727749 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727724    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:19.727865 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727758    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:19.727865 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727773    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:19.727865 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:19.727825    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:21.72780866 +0000 UTC m=+6.121496071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:20.056986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:20.056868    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:16:18 +0000 UTC" deadline="2028-05-23 13:12:07.158426006 +0000 UTC"
Sep 11 13:21:20.056986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:20.056912    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14879h50m47.101519304s"
Sep 11 13:21:20.242400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:20.242365    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:20.242599 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:20.242541    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:21.242826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:21.242746    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:21.243380 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.242891    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:21.243380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:21.243332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:21.243503 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.243440    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:21.644565 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:21.644509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:21.644765 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.644725    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:21.644827 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.644791    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:25.644772073 +0000 UTC m=+10.038459468 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:21.745457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:21.745404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:21.745682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:21.745501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:21.745768 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.745692    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:21.745768 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.745713    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:21.745768 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.745729    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:21.745929 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.745799    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:25.745779626 +0000 UTC m=+10.139467021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:21.746260 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.746236    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:21.746345 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:21.746300    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:25.746284143 +0000 UTC m=+10.139971550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:22.244756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:22.242798    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:22.244756 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:22.242960    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:23.243062 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:23.242304    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:23.243062 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:23.242449    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:23.243062 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:23.242905    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:23.243062 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:23.242994    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:24.242156 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:24.242095    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:24.242672 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:24.242259    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:25.242897 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:25.242850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:25.243309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:25.242850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:25.243309 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.243107    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:25.243467 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.243432    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:25.681129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:25.681010    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:25.681297 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.681176    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:25.681297 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.681256    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:33.681232647 +0000 UTC m=+18.074920053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:25.782214 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:25.782170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:25.782451 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:25.782271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:25.782451 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782421    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:25.782451 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782444    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:25.782640 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782458    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:25.782640 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782520    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:33.782502304 +0000 UTC m=+18.176189710 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:25.782640 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782598    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:25.782879 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:25.782673    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:33.782655302 +0000 UTC m=+18.176342708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:26.246034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:26.245981    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:26.246682 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:26.246125    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:27.242189 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:27.242146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:27.242367 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:27.242281    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:27.242584 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:27.242564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:27.242700 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:27.242676    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:28.245296 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:28.245267    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:28.245783 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:28.245397    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:29.242786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:29.242751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:29.242996 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:29.242869    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:29.242996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:29.242933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:29.243104 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:29.243052    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:30.242719 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:30.242684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:30.243088 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:30.242835    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:31.242159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:31.242103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:31.242337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:31.242104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:31.242337 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:31.242221    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:31.242456 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:31.242332    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:32.242425 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:32.242396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:32.242879 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:32.242550    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:33.242457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:33.242420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:33.242964 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:33.242422    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:33.242964 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.242538    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:33.242964 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.242615    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:33.744804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:33.744766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:33.744990 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.744927    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:33.745060 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.745019    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:21:49.744990583 +0000 UTC m=+34.138677986 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:33.845651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:33.845577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:33.845651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:33.845650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:33.845893 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845778    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:33.845893 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845813    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:33.845893 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845828    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:33.845893 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845786    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:33.846060 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845888    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:21:49.845869594 +0000 UTC m=+34.239557029 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:33.846060 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:33.845967    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:49.845948429 +0000 UTC m=+34.239635831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:34.242855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:34.242774    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:34.243273 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:34.242924    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:35.242980 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:35.242942    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:35.243390 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:35.243059    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:35.243390 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:35.243120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:35.243390 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:35.243216    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:36.243370 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:36.243323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:36.243780 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:36.243467    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:37.242564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:37.242541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:37.242627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:37.242569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:37.242696 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:37.242675    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:37.242793 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:37.242778    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:38.242368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.242186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:38.243025 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:38.242441    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:38.316604 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.316566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5" event={"ID":"b2e18965-023d-4944-acfc-dc7878dcf4f5","Type":"ContainerStarted","Data":"2c146b8c6ecb9b5fdcdf7cfbf7b475d4d3049519dc4cb35643dcc886fda998bb"}
Sep 11 13:21:38.317967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.317943    2576 generic.go:356] "Generic (PLEG): container finished" podID="c415eb3b64e424c5a88aa6d0ac326c37" containerID="c9922ed268e61f6c26404dcbde8874f18758cc67a225700b31a3bae12f23d7b2" exitCode=0
Sep 11 13:21:38.318080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.318003    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal" event={"ID":"c415eb3b64e424c5a88aa6d0ac326c37","Type":"ContainerDied","Data":"c9922ed268e61f6c26404dcbde8874f18758cc67a225700b31a3bae12f23d7b2"}
Sep 11 13:21:38.319317 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.319294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal" event={"ID":"e834e13edb4b1c98f5d9c9a73bedc2fb","Type":"ContainerStarted","Data":"28f7dd300dcd191ca788cbfc923e0b411d14f8d6ebce92b631d1d5c7a425718c"}
Sep 11 13:21:38.320763 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.320708    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f7lfj" event={"ID":"d883a3c1-c563-425c-a585-b04894de1190","Type":"ContainerStarted","Data":"b735e1ee5f60d4d3be7537aaac496511ad33ff03a7c218cf3dfb50579cf57f1f"}
Sep 11 13:21:38.323718 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"d37235d0c80402cfa56260c7ba1c2a9ee5fc2e6c2fcba99853482708b9d4ad5c"}
Sep 11 13:21:38.323805 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"3a7dc5f21dc8eca67c2007c860e07302f9effeab6b75a8258ed98f1a56ecf044"}
Sep 11 13:21:38.323869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"0a405f71695382e88f87a0034b462c3b6880d2d78e34cff570266c26f4738b55"}
Sep 11 13:21:38.323869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"eb1816b516a5a1f55b1eac0340638ceb73d54ffc0adf656ebee74c86323c6b7a"}
Sep 11 13:21:38.323977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323911    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"b02b82f12b0d2882d0f8aacae11230b8bef1738ed12c1350dc6cf391f768952d"}
Sep 11 13:21:38.323977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.323956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"136298cf0917331ea0476a7f4a7b6b2b938e0b0efdf1a5222562d19009f80c2e"}
Sep 11 13:21:38.324969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.324946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tq76r" event={"ID":"93b4f034-59d0-442e-b948-370d656307d9","Type":"ContainerStarted","Data":"fdf10adc9c61c1b1e2392fede93b48d018e640fd11807bdd678fcbbf2b6d8d48"}
Sep 11 13:21:38.326356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.326336    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbc8w" event={"ID":"a6ecaf2b-b9f6-4edf-8cf9-e561b48b56d8","Type":"ContainerStarted","Data":"424b09d5105445f1f9399e21e52474f063f4b94dcbd317b123c84f39a3615e69"}
Sep 11 13:21:38.327715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.327692    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="69ed28a40b3299b9cded916e3647000f0b0bc7f477a65daac1c59934680dbd47" exitCode=0
Sep 11 13:21:38.327806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.327768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"69ed28a40b3299b9cded916e3647000f0b0bc7f477a65daac1c59934680dbd47"}
Sep 11 13:21:38.329286 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.329265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tkfh5" event={"ID":"ec4d4872-4d83-4d4a-a78f-6945f2447a76","Type":"ContainerStarted","Data":"20c3832b1a7e0eccc58af2c132cf937026ad0ba6b04919c1002bf447d9600275"}
Sep 11 13:21:38.330693 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.330672    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nvmtf" event={"ID":"3fab8561-db61-40c9-9972-73e8e638cc7f","Type":"ContainerStarted","Data":"21e75476c5740adc8150868bd82327e0bafe3bcc80eeb5d1394420b93bce97b9"}
Sep 11 13:21:38.357610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.357565    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-221.ec2.internal" podStartSLOduration=20.357547534 podStartE2EDuration="20.357547534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.334852445 +0000 UTC m=+22.728539855" watchObservedRunningTime="2026-09-11 13:21:38.357547534 +0000 UTC m=+22.751234945"
Sep 11 13:21:38.375217 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.375176    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qbc8w" podStartSLOduration=2.525618737 podStartE2EDuration="21.375163049s" totalImagesPullingTime="18.849544312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.374930683 +0000 UTC m=+22.768618093" watchObservedRunningTime="2026-09-11 13:21:38.375163049 +0000 UTC m=+22.768850459"
Sep 11 13:21:38.389939 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.389891    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tq76r" podStartSLOduration=2.558856007 podStartE2EDuration="21.389878594s" totalImagesPullingTime="18.831022587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.389378057 +0000 UTC m=+22.783065468" watchObservedRunningTime="2026-09-11 13:21:38.389878594 +0000 UTC m=+22.783566004"
Sep 11 13:21:38.436344 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.436292    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nvmtf" podStartSLOduration=10.069563412 podStartE2EDuration="21.436271565s" totalImagesPullingTime="11.366708153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.40538284 +0000 UTC m=+22.799070289" watchObservedRunningTime="2026-09-11 13:21:38.436271565 +0000 UTC m=+22.829958976"
Sep 11 13:21:38.453150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.453106    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f7lfj" podStartSLOduration=2.6550337920000002 podStartE2EDuration="21.453091015s" totalImagesPullingTime="18.798057223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.435981281 +0000 UTC m=+22.829668690" watchObservedRunningTime="2026-09-11 13:21:38.453091015 +0000 UTC m=+22.846778419"
Sep 11 13:21:38.453887 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.453846    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tkfh5" podStartSLOduration=2.676869731 podStartE2EDuration="21.453831296s" totalImagesPullingTime="18.776961565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:38.453214147 +0000 UTC m=+22.846901558" watchObservedRunningTime="2026-09-11 13:21:38.453831296 +0000 UTC m=+22.847518711"
Sep 11 13:21:38.973562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:38.973540    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:21:39.196820 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.196660    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:21:38.973558149Z","UUID":"60911942-4555-4cfb-8a84-7b038636d953","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:21:39.198230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.198201    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:21:39.198230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.198235    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:21:39.227918 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.227887    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:39.228437 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.228412    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:39.242352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.242326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:39.242352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.242352    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:39.242985 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:39.242436    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:39.242985 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:39.242562    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:39.335420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.335377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qqjvw" event={"ID":"17d15b5c-ceaa-4092-96ad-c48f5ae808cd","Type":"ContainerStarted","Data":"08a5164ddb8f48a1aa506ebf7655826ea92e33538d31a83d90001f17c5ef6b95"}
Sep 11 13:21:39.337558 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.337525    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5" event={"ID":"b2e18965-023d-4944-acfc-dc7878dcf4f5","Type":"ContainerStarted","Data":"dd8314515e1f0515d1d2fcb0c697dde450e1c871b0ec42e600210123a933a3c4"}
Sep 11 13:21:39.351855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:39.351811    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qqjvw" podStartSLOduration=3.533467275 podStartE2EDuration="22.351795817s" totalImagesPullingTime="18.818328542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:39.35160054 +0000 UTC m=+23.745287950" watchObservedRunningTime="2026-09-11 13:21:39.351795817 +0000 UTC m=+23.745483225"
Sep 11 13:21:40.243040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.242771    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:40.243476 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:40.243167    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:40.342397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.342357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5" event={"ID":"b2e18965-023d-4944-acfc-dc7878dcf4f5","Type":"ContainerStarted","Data":"8432ea6332ac156402a24353ba27c2e569a0da56fc678dc3cd6b34aec86a3f03"}
Sep 11 13:21:40.344449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.344421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal" event={"ID":"c415eb3b64e424c5a88aa6d0ac326c37","Type":"ContainerStarted","Data":"5b850a60e891593af403052730153ae10aeaf2e130d8bed4d9ee0e127a03327d"}
Sep 11 13:21:40.348451 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.348428    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:21:40.348593 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.348453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"9ae754b9856baec754f8f1d925b35aece9f0b2c3bd6840f656fab4dcac1e2082"}
Sep 11 13:21:40.401856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:40.401757    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sp7s5" podStartSLOduration=2.647433521 podStartE2EDuration="23.401740778s" totalImagesPullingTime="20.754307257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:40.379058611 +0000 UTC m=+24.772746023" watchObservedRunningTime="2026-09-11 13:21:40.401740778 +0000 UTC m=+24.795428188"
Sep 11 13:21:41.242070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:41.242037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:41.242270 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:41.242141    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:41.242270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:41.242195    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:41.242360 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:41.242282    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:42.242285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:42.242253    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:42.242781 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:42.242390    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:43.242875 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:43.242843    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:43.243242 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:43.242943    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:43.243242 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:43.242972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:43.243242 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:43.243031    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:43.359685 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:43.359598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" event={"ID":"494d8db9-4794-4883-a7f4-08f69f46dd19","Type":"ContainerStarted","Data":"4337eaaf8add119f4636739600f46e790c84454a57d9e901c875182a93c66739"}
Sep 11 13:21:44.242927 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.242750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:44.243310 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:44.243015    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:44.363989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.363954    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="95c18dee7d0e6fe1d2f31761b5bfa18e6735d2f7018864cfaf8701e188f52da7" exitCode=0
Sep 11 13:21:44.364186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.364022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"95c18dee7d0e6fe1d2f31761b5bfa18e6735d2f7018864cfaf8701e188f52da7"}
Sep 11 13:21:44.364282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.364261    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:44.364351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.364337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:44.364398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.364392    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:44.379747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.379723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:44.379891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.379852    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:21:44.404067 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:44.404020    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-221.ec2.internal" podStartSLOduration=26.404006961 podStartE2EDuration="26.404006961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:40.401392225 +0000 UTC m=+24.795079636" watchObservedRunningTime="2026-09-11 13:21:44.404006961 +0000 UTC m=+28.797694371"
Sep 11 13:21:45.107345 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.107311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:45.107480 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.107454    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 13:21:45.108172 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.108151    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nvmtf"
Sep 11 13:21:45.124940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.124883    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9" podStartSLOduration=8.912549665 podStartE2EDuration="28.124857779s" totalImagesPullingTime="19.212308114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:44.444552424 +0000 UTC m=+28.838239834" watchObservedRunningTime="2026-09-11 13:21:45.124857779 +0000 UTC m=+29.518545191"
Sep 11 13:21:45.242732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.242697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:45.242732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.242726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:45.242907 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:45.242794    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:45.242945 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:45.242908    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:45.364592 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.364485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-crjpf"]
Sep 11 13:21:45.367169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.367144    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rgwvv"]
Sep 11 13:21:45.367308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.367266    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:45.367414 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:45.367395    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:45.369341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.369312    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="c4ad835421ec20e5f83410d1e221e38478e8c7f89970de0dfafc618ec4e26835" exitCode=0
Sep 11 13:21:45.369453 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.369359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"c4ad835421ec20e5f83410d1e221e38478e8c7f89970de0dfafc618ec4e26835"}
Sep 11 13:21:45.369453 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.369386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:45.369654 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:45.369629    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:45.371170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.370743    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vdlnp"]
Sep 11 13:21:45.371170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:45.370810    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:45.371170 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:45.370909    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:46.374326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:46.374292    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="120a06c09fe202f131820ceb320caddb63ca2886dcbe816f50798b809b90a2e6" exitCode=0
Sep 11 13:21:46.374768 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:46.374378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"120a06c09fe202f131820ceb320caddb63ca2886dcbe816f50798b809b90a2e6"}
Sep 11 13:21:47.243189 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:47.242960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:47.243378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:47.242971    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:47.243378 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:47.243283    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:47.243476 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:47.243381    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:47.243476 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:47.243016    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:47.243577 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:47.243532    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:49.243051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.243017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:49.243383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.243038    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:49.243383 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.243136    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:49.243383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.243145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:49.243383 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.243240    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:49.243383 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.243305    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:49.762136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.762107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:49.762303 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.762225    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:49.762303 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.762293    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:22:21.76227241 +0000 UTC m=+66.155959800 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:21:49.863357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.863327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:49.863536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:49.863381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:49.863536 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863507    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:21:49.863536 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863534    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:21:49.863653 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863546    2576 projected.go:196] Error preparing data for projected volume kube-api-access-wghzm for pod openshift-network-diagnostics/network-check-target-vdlnp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:49.863653 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863597    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm podName:99e53806-cdfb-4c9c-a149-c8d7a2a2994b nodeName:}" failed. No retries permitted until 2026-09-11 13:22:21.863578667 +0000 UTC m=+66.257266069 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wghzm" (UniqueName: "kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm") pod "network-check-target-vdlnp" (UID: "99e53806-cdfb-4c9c-a149-c8d7a2a2994b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:21:49.863653 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863490    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:49.863653 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:49.863636    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret podName:45c1f894-db5d-445d-a062-41f989520fa2 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:21.863625684 +0000 UTC m=+66.257313075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret") pod "global-pull-secret-syncer-crjpf" (UID: "45c1f894-db5d-445d-a062-41f989520fa2") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:21:51.242128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.242047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:51.242128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.242078    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:51.242128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.242092    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:51.242719 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.242206    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:21:51.242719 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.242329    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vdlnp" podUID="99e53806-cdfb-4c9c-a149-c8d7a2a2994b"
Sep 11 13:21:51.242719 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.242461    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-crjpf" podUID="45c1f894-db5d-445d-a062-41f989520fa2"
Sep 11 13:21:51.356585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.356555    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-221.ec2.internal" event="NodeReady"
Sep 11 13:21:51.356764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.356710    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:21:51.414026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.413985    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"]
Sep 11 13:21:51.446481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.446444    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:21:51.446672 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.446595    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.453945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.453914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 13:21:51.454070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.453972    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 13:21:51.454253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.454237    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 13:21:51.459192 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.458417    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"]
Sep 11 13:21:51.459192 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.458951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.462449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.461353    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:21:51.462449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.461615    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 13:21:51.462449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.462238    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:21:51.462449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.462245    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:21:51.462449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.462369    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zq2ct\""
Sep 11 13:21:51.474324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.474300    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"]
Sep 11 13:21:51.474513 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.474437    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.476248 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.476229    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:21:51.476694 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.476672    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-v9fmg\""
Sep 11 13:21:51.478361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.478345    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 13:21:51.492408 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492359    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"]
Sep 11 13:21:51.492408 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492402    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"]
Sep 11 13:21:51.492532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492419    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"]
Sep 11 13:21:51.492532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492437    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:21:51.492532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492461    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cn9lc"]
Sep 11 13:21:51.492532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.492470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.498248 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.498046    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 13:21:51.498248 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.498074    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 13:21:51.498389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.498312    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 13:21:51.498389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.498312    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 13:21:51.509141 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.509118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nllgk"]
Sep 11 13:21:51.509266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.509154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.514305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.514158    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:21:51.514305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.514182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:21:51.514305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.514198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vqfbt\""
Sep 11 13:21:51.524575 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.524552    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn9lc"]
Sep 11 13:21:51.524676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.524602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nllgk"]
Sep 11 13:21:51.524731 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.524676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.529162 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.529138    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ng6x\""
Sep 11 13:21:51.529452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.529435    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:21:51.530034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.530013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:21:51.530129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.530041    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:21:51.576356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.576531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.576531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449d1056-0223-44d7-a6d9-fb500f0a1935-tmp\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.576531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2gl2\" (UniqueName: \"kubernetes.io/projected/449d1056-0223-44d7-a6d9-fb500f0a1935-kube-api-access-n2gl2\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.576531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576531    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/427e42e5-9622-4b3c-b14a-ce64a6f622ed-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576550    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb4v4\" (UniqueName: \"kubernetes.io/projected/8b5cbd93-22a6-4a03-9813-880f21dcf37d-kube-api-access-pb4v4\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576705    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj644\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/449d1056-0223-44d7-a6d9-fb500f0a1935-klusterlet-config\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.576894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5cbd93-22a6-4a03-9813-880f21dcf37d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.577153 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.576914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwf4f\" (UniqueName: \"kubernetes.io/projected/427e42e5-9622-4b3c-b14a-ce64a6f622ed-kube-api-access-mwf4f\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.666445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.666403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:51.677446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677423    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.677582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.677582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449d1056-0223-44d7-a6d9-fb500f0a1935-tmp\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.677582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2gl2\" (UniqueName: \"kubernetes.io/projected/449d1056-0223-44d7-a6d9-fb500f0a1935-kube-api-access-n2gl2\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.677582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.677582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/427e42e5-9622-4b3c-b14a-ce64a6f622ed-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677590    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb4v4\" (UniqueName: \"kubernetes.io/projected/8b5cbd93-22a6-4a03-9813-880f21dcf37d-kube-api-access-pb4v4\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-tmp-dir\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-config-volume\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4b75c12-9d04-448a-9285-b02523dd697e-cert\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.677830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.678335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrmkp\" (UniqueName: \"kubernetes.io/projected/f4b75c12-9d04-448a-9285-b02523dd697e-kube-api-access-vrmkp\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.678335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677863    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.678335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/449d1056-0223-44d7-a6d9-fb500f0a1935-tmp\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.678335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.677883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj644\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.678530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.678486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/427e42e5-9622-4b3c-b14a-ce64a6f622ed-additional-service-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.679233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.678892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.679233 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.678988    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:21:51.679233 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.679003    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:21:51.679233 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.679065    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:52.179047894 +0000 UTC m=+36.572735300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzngc\" (UniqueName: \"kubernetes.io/projected/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-kube-api-access-vzngc\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/449d1056-0223-44d7-a6d9-fb500f0a1935-klusterlet-config\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5cbd93-22a6-4a03-9813-880f21dcf37d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.679535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.679478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwf4f\" (UniqueName: \"kubernetes.io/projected/427e42e5-9622-4b3c-b14a-ce64a6f622ed-kube-api-access-mwf4f\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.680343 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.680312    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.682794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.682697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.682794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.682709    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.682794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.682742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.682794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.682777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.683032 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.682781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-ca\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.683199 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.683177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/427e42e5-9622-4b3c-b14a-ce64a6f622ed-hub\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.683264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.683239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8b5cbd93-22a6-4a03-9813-880f21dcf37d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.683359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.683340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/449d1056-0223-44d7-a6d9-fb500f0a1935-klusterlet-config\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.687164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.687141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2gl2\" (UniqueName: \"kubernetes.io/projected/449d1056-0223-44d7-a6d9-fb500f0a1935-kube-api-access-n2gl2\") pod \"klusterlet-addon-workmgr-68ddbdf896-j78ht\" (UID: \"449d1056-0223-44d7-a6d9-fb500f0a1935\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.687259 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.687143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb4v4\" (UniqueName: \"kubernetes.io/projected/8b5cbd93-22a6-4a03-9813-880f21dcf37d-kube-api-access-pb4v4\") pod \"managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7\" (UID: \"8b5cbd93-22a6-4a03-9813-880f21dcf37d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.688263 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.688239    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwf4f\" (UniqueName: \"kubernetes.io/projected/427e42e5-9622-4b3c-b14a-ce64a6f622ed-kube-api-access-mwf4f\") pod \"cluster-proxy-proxy-agent-97768c57c-ntp99\" (UID: \"427e42e5-9622-4b3c-b14a-ce64a6f622ed\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.688550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.688534    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.689927 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.689906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj644\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:51.758933 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.758860    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:21:51.780217 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.780324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780278    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-tmp-dir\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.780324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-config-volume\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.780324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4b75c12-9d04-448a-9285-b02523dd697e-cert\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.780472 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.780357    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:51.780472 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:51.780446    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:52.280423845 +0000 UTC m=+36.674111251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:51.780602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrmkp\" (UniqueName: \"kubernetes.io/projected/f4b75c12-9d04-448a-9285-b02523dd697e-kube-api-access-vrmkp\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.780661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzngc\" (UniqueName: \"kubernetes.io/projected/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-kube-api-access-vzngc\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.780713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.780683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-tmp-dir\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.781025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.781005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-config-volume\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.785385 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.785363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f4b75c12-9d04-448a-9285-b02523dd697e-cert\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.786915 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.786886    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:51.789698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.789682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrmkp\" (UniqueName: \"kubernetes.io/projected/f4b75c12-9d04-448a-9285-b02523dd697e-kube-api-access-vrmkp\") pod \"ingress-canary-nllgk\" (UID: \"f4b75c12-9d04-448a-9285-b02523dd697e\") " pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:51.793120 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.793101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzngc\" (UniqueName: \"kubernetes.io/projected/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-kube-api-access-vzngc\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:51.796015 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.795999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"
Sep 11 13:21:51.804528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.804506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:21:51.839848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:51.839824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nllgk"
Sep 11 13:21:52.144960 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.144694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"]
Sep 11 13:21:52.186066 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.186036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:52.186225 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:52.186169    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:21:52.186225 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:52.186188    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:21:52.186336 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:52.186245    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:53.18622802 +0000 UTC m=+37.579915414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:21:52.207175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.207147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7"]
Sep 11 13:21:52.216380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.216352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nllgk"]
Sep 11 13:21:52.220663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.220577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"]
Sep 11 13:21:52.232583 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:52.232553    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b5cbd93_22a6_4a03_9813_880f21dcf37d.slice/crio-3be64c4b4c37ca7a26ab1956ab5850443173e05795c99dd1bb712333f4fab8b5 WatchSource:0}: Error finding container 3be64c4b4c37ca7a26ab1956ab5850443173e05795c99dd1bb712333f4fab8b5: Status 404 returned error can't find the container with id 3be64c4b4c37ca7a26ab1956ab5850443173e05795c99dd1bb712333f4fab8b5
Sep 11 13:21:52.232838 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:52.232811    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod427e42e5_9622_4b3c_b14a_ce64a6f622ed.slice/crio-7eb8a6d0a880bfc0b6d90a827c3364c01d89fa1511edd6484a5d23618585f656 WatchSource:0}: Error finding container 7eb8a6d0a880bfc0b6d90a827c3364c01d89fa1511edd6484a5d23618585f656: Status 404 returned error can't find the container with id 7eb8a6d0a880bfc0b6d90a827c3364c01d89fa1511edd6484a5d23618585f656
Sep 11 13:21:52.233180 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:52.233149    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b75c12_9d04_448a_9285_b02523dd697e.slice/crio-27f889fff0d0d8f4b56069ec3c214f5e60ff5bb57887038e61f51f3badaac307 WatchSource:0}: Error finding container 27f889fff0d0d8f4b56069ec3c214f5e60ff5bb57887038e61f51f3badaac307: Status 404 returned error can't find the container with id 27f889fff0d0d8f4b56069ec3c214f5e60ff5bb57887038e61f51f3badaac307
Sep 11 13:21:52.233552 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:21:52.233530    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449d1056_0223_44d7_a6d9_fb500f0a1935.slice/crio-e97cb8e3bb39ba42171045849505a1e4066845b05af8a29d68a7c777ea9911ff WatchSource:0}: Error finding container e97cb8e3bb39ba42171045849505a1e4066845b05af8a29d68a7c777ea9911ff: Status 404 returned error can't find the container with id e97cb8e3bb39ba42171045849505a1e4066845b05af8a29d68a7c777ea9911ff
Sep 11 13:21:52.287216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.287147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:52.287561 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:52.287289    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:52.287561 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:52.287343    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:53.287329613 +0000 UTC m=+37.681017021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:52.402403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.402364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" event={"ID":"8b5cbd93-22a6-4a03-9813-880f21dcf37d","Type":"ContainerStarted","Data":"3be64c4b4c37ca7a26ab1956ab5850443173e05795c99dd1bb712333f4fab8b5"}
Sep 11 13:21:52.403381 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.403352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" event={"ID":"449d1056-0223-44d7-a6d9-fb500f0a1935","Type":"ContainerStarted","Data":"e97cb8e3bb39ba42171045849505a1e4066845b05af8a29d68a7c777ea9911ff"}
Sep 11 13:21:52.404452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.404430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerStarted","Data":"7eb8a6d0a880bfc0b6d90a827c3364c01d89fa1511edd6484a5d23618585f656"}
Sep 11 13:21:52.405536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.405512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nllgk" event={"ID":"f4b75c12-9d04-448a-9285-b02523dd697e","Type":"ContainerStarted","Data":"27f889fff0d0d8f4b56069ec3c214f5e60ff5bb57887038e61f51f3badaac307"}
Sep 11 13:21:52.408314 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.408294    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="44b4b5f40bcc86b5033297d45190d90c59f242485de32230e311b5970febbe6d" exitCode=0
Sep 11 13:21:52.408397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:52.408324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"44b4b5f40bcc86b5033297d45190d90c59f242485de32230e311b5970febbe6d"}
Sep 11 13:21:53.195004 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.194968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:53.195262 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:53.195244    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:21:53.195344 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:53.195266    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:21:53.195344 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:53.195327    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:55.195308581 +0000 UTC m=+39.588995973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:21:53.242449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.242379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:21:53.242771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.242750    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:21:53.243183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.243165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:21:53.248110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248075    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:21:53.248110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:21:53.248337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248319    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2l67\""
Sep 11 13:21:53.248400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tkfcx\""
Sep 11 13:21:53.248448 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248424    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:21:53.248828 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.248787    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:21:53.295976 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.295944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:53.296724 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:53.296096    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:53.296724 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:53.296159    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:55.296140828 +0000 UTC m=+39.689828233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:53.418458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.418241    2576 generic.go:356] "Generic (PLEG): container finished" podID="a2949ce6-3bb6-4d10-b03c-0fb47be3bbea" containerID="26ea51e5db19719ac5b8231aabd142c8461992cf52b13a8b98a601b6da6acf29" exitCode=0
Sep 11 13:21:53.418458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:53.418288    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerDied","Data":"26ea51e5db19719ac5b8231aabd142c8461992cf52b13a8b98a601b6da6acf29"}
Sep 11 13:21:54.439550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:54.439509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" event={"ID":"a2949ce6-3bb6-4d10-b03c-0fb47be3bbea","Type":"ContainerStarted","Data":"aff2d352258fe7a64c8f80981bc0d037c728191507cc90e391fbb30c99c66e2d"}
Sep 11 13:21:54.470246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:54.470173    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ck5bd" podStartSLOduration=6.537906093 podStartE2EDuration="37.470150087s" totalImagesPullingTime="30.932243994s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:54.469700803 +0000 UTC m=+38.863388215" watchObservedRunningTime="2026-09-11 13:21:54.470150087 +0000 UTC m=+38.863837500"
Sep 11 13:21:54.662407 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:54.662266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:54.742477 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:54.742436    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.216194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.216157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:55.216354 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:55.216328    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:21:55.216354 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:55.216345    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:21:55.216451 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:55.216415    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:59.216396069 +0000 UTC m=+43.610083460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:21:55.317920 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.317145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:55.317920 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:55.317307    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:55.317920 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:55.317517    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:21:59.317456494 +0000 UTC m=+43.711143901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:21:55.428030 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.427994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.537076 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.537037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.659490 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.659442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.762876 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.762809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.860653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.860561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:55.956633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:55.956542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:57.520757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:57.520680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:57.606808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:57.606771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:58.330439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:58.330402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:58.452207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:58.452149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:58.797019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:58.796971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:58.891978 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:58.891934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:21:59.252875 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:59.252781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:21:59.253053 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:59.252945    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:21:59.253053 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:59.252968    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:21:59.253053 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:59.253039    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:07.253018833 +0000 UTC m=+51.646706235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:21:59.353631 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:21:59.353594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:21:59.353790 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:59.353759    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:21:59.353873 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:21:59.353852    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:07.353829082 +0000 UTC m=+51.747516476 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:00.435183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:00.435148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:01.459753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.459729    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" event={"ID":"8b5cbd93-22a6-4a03-9813-880f21dcf37d","Type":"ContainerStarted","Data":"b4500569e2afbe02feab3d40befcb9cd1113f3f303d509257bba59d6fd8f714c"}
Sep 11 13:22:01.461222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.461200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" event={"ID":"449d1056-0223-44d7-a6d9-fb500f0a1935","Type":"ContainerStarted","Data":"fc254eca1c242f359f54a6c00f3aeb52e9fde4e79599b7352705c14e3faf2ee3"}
Sep 11 13:22:01.461324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.461253    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:22:01.462968 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.462949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:22:01.463198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.463180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerStarted","Data":"d674f1966ef47a3cfaaab350fc2a62941d73e03c4508bed34bbd3e0f6a20fb13"}
Sep 11 13:22:01.463273 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.463206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerStarted","Data":"28a6f5554603bbbf66f87e5cbdf1f2faf4c6bb1b03cea5d9709c94dc93a20b02"}
Sep 11 13:22:01.464562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.464538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nllgk" event={"ID":"f4b75c12-9d04-448a-9285-b02523dd697e","Type":"ContainerStarted","Data":"a9688bbe0fc3c18614d91e42992d258e38f6a816a7023a598d7f6c346fef9bd6"}
Sep 11 13:22:01.520336 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.520275    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" podStartSLOduration=33.652123261 podStartE2EDuration="42.520256921s" totalImagesPullingTime="8.86813366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:01.488446739 +0000 UTC m=+45.882134151" watchObservedRunningTime="2026-09-11 13:22:01.520256921 +0000 UTC m=+45.913944329"
Sep 11 13:22:01.543185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.543114    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nllgk" podStartSLOduration=1.67339901 podStartE2EDuration="10.543095936s" totalImagesPullingTime="8.869696926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:01.519882868 +0000 UTC m=+45.913570280" watchObservedRunningTime="2026-09-11 13:22:01.543095936 +0000 UTC m=+45.936783349"
Sep 11 13:22:01.543587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:01.543556    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" podStartSLOduration=33.659488951 podStartE2EDuration="42.543548137s" totalImagesPullingTime="8.884059186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:01.543253751 +0000 UTC m=+45.936941144" watchObservedRunningTime="2026-09-11 13:22:01.543548137 +0000 UTC m=+45.937235547"
Sep 11 13:22:02.469378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:02.469347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerStarted","Data":"274b8b70b7c6b90aaaf83ee553d7db0b5aeee8be0c32193b1ac51966ae11c686"}
Sep 11 13:22:02.501313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:02.501259    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" podStartSLOduration=34.631824841 podStartE2EDuration="43.501241424s" totalImagesPullingTime="8.869416583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:02.500913724 +0000 UTC m=+46.894601137" watchObservedRunningTime="2026-09-11 13:22:02.501241424 +0000 UTC m=+46.894928835"
Sep 11 13:22:04.077159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:04.077117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:07.304670 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:07.304630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:22:07.305080 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:07.304773    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:22:07.305080 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:07.304794    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:22:07.305080 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:07.304848    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:23.304833482 +0000 UTC m=+67.698520870 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:22:07.404952 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:07.404917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:22:07.405127 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:07.405078    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:07.405174 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:07.405156    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:23.405140883 +0000 UTC m=+67.798828277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:09.159220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:09.159177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:09.254221 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:09.254167    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:11.674371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:11.674335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:11.773403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:11.773372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:11.847722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:11.847690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:13.399142 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:13.399112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:16.385012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:16.384974    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wz6r9"
Sep 11 13:22:16.560098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:16.560068    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:16.620545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:16.620516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:21.725613 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.725585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:21.814389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.814356    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:22:21.817934 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.817917    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:22:21.825540 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:21.825526    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:22:21.825603 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:21.825579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:23:25.82556428 +0000 UTC m=+130.219251668 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : secret "metrics-daemon-secret" not found
Sep 11 13:22:21.827775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.827753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:21.915149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.915116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:22:21.915274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.915172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:22:21.917903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.917886    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:22:21.918397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.918378    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:22:21.928316 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.928290    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:22:21.928638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.928621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/45c1f894-db5d-445d-a062-41f989520fa2-original-pull-secret\") pod \"global-pull-secret-syncer-crjpf\" (UID: \"45c1f894-db5d-445d-a062-41f989520fa2\") " pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:22:21.938610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:21.938588    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wghzm\" (UniqueName: \"kubernetes.io/projected/99e53806-cdfb-4c9c-a149-c8d7a2a2994b-kube-api-access-wghzm\") pod \"network-check-target-vdlnp\" (UID: \"99e53806-cdfb-4c9c-a149-c8d7a2a2994b\") " pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:22:22.073302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.073268    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tkfcx\""
Sep 11 13:22:22.078204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.078188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-crjpf"
Sep 11 13:22:22.081324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.081307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:22:22.376854 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.376785    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-crjpf"]
Sep 11 13:22:22.379846 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:22:22.379823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c1f894_db5d_445d_a062_41f989520fa2.slice/crio-4f5610d1c895f70ee054796a6f022ffa2f0a35976be6294d4a8b77ac9f867d9e WatchSource:0}: Error finding container 4f5610d1c895f70ee054796a6f022ffa2f0a35976be6294d4a8b77ac9f867d9e: Status 404 returned error can't find the container with id 4f5610d1c895f70ee054796a6f022ffa2f0a35976be6294d4a8b77ac9f867d9e
Sep 11 13:22:22.404651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.404629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vdlnp"]
Sep 11 13:22:22.407340 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:22:22.407320    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99e53806_cdfb_4c9c_a149_c8d7a2a2994b.slice/crio-debee2eb0c4415b476a989fcabf23bdac4b6657d5046b467757082501175e410 WatchSource:0}: Error finding container debee2eb0c4415b476a989fcabf23bdac4b6657d5046b467757082501175e410: Status 404 returned error can't find the container with id debee2eb0c4415b476a989fcabf23bdac4b6657d5046b467757082501175e410
Sep 11 13:22:22.529990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.529954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vdlnp" event={"ID":"99e53806-cdfb-4c9c-a149-c8d7a2a2994b","Type":"ContainerStarted","Data":"debee2eb0c4415b476a989fcabf23bdac4b6657d5046b467757082501175e410"}
Sep 11 13:22:22.531127 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:22.531106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-crjpf" event={"ID":"45c1f894-db5d-445d-a062-41f989520fa2","Type":"ContainerStarted","Data":"4f5610d1c895f70ee054796a6f022ffa2f0a35976be6294d4a8b77ac9f867d9e"}
Sep 11 13:22:23.287788 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:23.287751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:23.327078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:23.327033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:22:23.327263 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:23.327195    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:22:23.327263 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:23.327216    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:22:23.327378 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:23.327288    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:55.327267408 +0000 UTC m=+99.720954817 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:22:23.388961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:23.388923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:23.428411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:23.428362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:22:23.428611 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:23.428513    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:23.428611 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:23.428581    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:22:55.428562426 +0000 UTC m=+99.822249829 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:24.573891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:24.573849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:27.549621 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:27.549592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-crjpf" event={"ID":"45c1f894-db5d-445d-a062-41f989520fa2","Type":"ContainerStarted","Data":"a8cb233e0140a93c3cc863b9db738f95ce7720cc376d81cbc490c3061b43961c"}
Sep 11 13:22:27.550940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:27.550919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vdlnp" event={"ID":"99e53806-cdfb-4c9c-a149-c8d7a2a2994b","Type":"ContainerStarted","Data":"05348b12b18f8f6a6d7bc1bc153190b4ee8064fb8b11e1fd37bf8cfb0506d24f"}
Sep 11 13:22:27.551052 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:27.551042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:22:27.569881 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:27.569841    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-crjpf" podStartSLOduration=65.759538308 podStartE2EDuration="1m10.569827866s" totalImagesPullingTime="4.810289558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:27.568053778 +0000 UTC m=+71.961741189" watchObservedRunningTime="2026-09-11 13:22:27.569827866 +0000 UTC m=+71.963515277"
Sep 11 13:22:27.592802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:27.592761    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vdlnp" podStartSLOduration=65.813537066 podStartE2EDuration="1m10.592751073s" totalImagesPullingTime="4.779214007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:27.592347056 +0000 UTC m=+71.986034467" watchObservedRunningTime="2026-09-11 13:22:27.592751073 +0000 UTC m=+71.986438483"
Sep 11 13:22:33.628650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:33.628588    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:35.408242 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:35.408211    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:22:55.365119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:55.365060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:22:55.365573 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:55.365219    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:22:55.365573 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:55.365240    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5964d6c969-gsk2v: secret "image-registry-tls" not found
Sep 11 13:22:55.365573 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:55.365294    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls podName:563ccbae-9f03-4fe5-b582-568955abc290 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:59.365278914 +0000 UTC m=+163.758966303 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls") pod "image-registry-5964d6c969-gsk2v" (UID: "563ccbae-9f03-4fe5-b582-568955abc290") : secret "image-registry-tls" not found
Sep 11 13:22:55.465438 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:55.465402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:22:55.465638 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:55.465537    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:22:55.465638 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:22:55.465601    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls podName:5a39d72e-6a80-4205-93a5-8dd0567ba6b1 nodeName:}" failed. No retries permitted until 2026-09-11 13:23:59.465587576 +0000 UTC m=+163.859274964 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls") pod "dns-default-cn9lc" (UID: "5a39d72e-6a80-4205-93a5-8dd0567ba6b1") : secret "dns-default-metrics-tls" not found
Sep 11 13:22:58.556330 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:22:58.556297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vdlnp"
Sep 11 13:23:01.933555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:01.933523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:02.035609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:02.035571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:05.894922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.894885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:05.966679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.966607    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fds4p"]
Sep 11 13:23:05.970013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.969992    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:05.972381 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.972362    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:23:05.972658 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.972644    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:23:05.973277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.973260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:23:05.973277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.973272    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:23:05.973277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.973274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd6rn\""
Sep 11 13:23:05.983575 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:05.983554    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fds4p"]
Sep 11 13:23:06.056107 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.056082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:06.143358 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.143336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0da8220f-d89e-4f9a-a453-75b2a0947c5d-data-volume\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.143440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.143389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.143440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.143409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.143440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.143437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdgs8\" (UniqueName: \"kubernetes.io/projected/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-api-access-zdgs8\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.143587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.143457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0da8220f-d89e-4f9a-a453-75b2a0947c5d-crio-socket\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0da8220f-d89e-4f9a-a453-75b2a0947c5d-data-volume\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244240 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.244169    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.244240 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.244190    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.244390 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdgs8\" (UniqueName: \"kubernetes.io/projected/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-api-access-zdgs8\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244390 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.244245    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:06.744230483 +0000 UTC m=+111.137917876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.244390 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.244282    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:06.744261284 +0000 UTC m=+111.137948675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.244390 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0da8220f-d89e-4f9a-a453-75b2a0947c5d-crio-socket\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244758 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0da8220f-d89e-4f9a-a453-75b2a0947c5d-crio-socket\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.244898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.244398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0da8220f-d89e-4f9a-a453-75b2a0947c5d-data-volume\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.257010 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.256987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdgs8\" (UniqueName: \"kubernetes.io/projected/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-api-access-zdgs8\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.748106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.748077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.748274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:06.748167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:06.748274 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.748224    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.748355 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.748286    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:07.74827232 +0000 UTC m=+112.141959710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:06.748355 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.748228    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:06.748447 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:06.748377    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:07.748365439 +0000 UTC m=+112.142052829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:07.741455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:07.741426    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tkfh5_ec4d4872-4d83-4d4a-a78f-6945f2447a76/dns-node-resolver/0.log"
Sep 11 13:23:07.754724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:07.754699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:07.754794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:07.754733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:07.754830 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:07.754816    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:07.754867 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:07.754840    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:07.754900 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:07.754889    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.754873539 +0000 UTC m=+114.148560929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:07.754940 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:07.754904    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:09.75489897 +0000 UTC m=+114.148586358 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:07.920775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:07.920747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:08.724877 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:08.724852    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tq76r_93b4f034-59d0-442e-b948-370d656307d9/node-ca/0.log"
Sep 11 13:23:09.527474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:09.527445    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nllgk_f4b75c12-9d04-448a-9285-b02523dd697e/serve-healthcheck-canary/0.log"
Sep 11 13:23:09.769919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:09.769876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:09.769919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:09.769921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:09.770129 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:09.770003    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:09.770129 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:09.770021    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:09.770129 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:09.770069    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:13.770050526 +0000 UTC m=+118.163737916 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:09.770129 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:09.770085    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:13.770078479 +0000 UTC m=+118.163765868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:13.801176 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:13.801140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:13.801176 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:13.801179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:13.801657 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:13.801258    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:23:13.801657 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:13.801310    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:13.801657 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:13.801336    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:21.801320189 +0000 UTC m=+126.195007578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : configmap "kube-rbac-proxy" not found
Sep 11 13:23:13.801657 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:13.801376    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:21.801359677 +0000 UTC m=+126.195047085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:14.588858 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:14.588824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:14.683106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:14.683074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:14.783379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:14.783346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.118458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.118421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.222723 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.222690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.530908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.530880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.631770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.631742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.797861 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.797791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.894488 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.894458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:16.994806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:16.994775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:18.441685 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:18.441657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:18.513594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:18.513564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:18.974595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:18.974567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.179576 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.179543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.282268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.282228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.380025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.379991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.482161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.482082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.536648 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.536619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:20.890168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:20.890138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:21.863557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:21.863519    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:21.864053 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:21.863570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:21.864053 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:21.863689    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:21.864053 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:21.863750    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls podName:0da8220f-d89e-4f9a-a453-75b2a0947c5d nodeName:}" failed. No retries permitted until 2026-09-11 13:23:37.863732031 +0000 UTC m=+142.257419424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fds4p" (UID: "0da8220f-d89e-4f9a-a453-75b2a0947c5d") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:23:21.864214 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:21.864137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0da8220f-d89e-4f9a-a453-75b2a0947c5d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:22.101804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:22.101773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:22.288169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:22.288139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:22.494173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:22.494138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:22.682999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:22.682925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:22.875925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:22.875893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:23.477206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:23.477173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:24.076529 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:24.076484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:24.191202 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:24.191170    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:24.478842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:24.478758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:24.544364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:24.544328    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:24.670691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:24.670662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.268534 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:25.268488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.323482 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:25.323452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.384598 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:25.384577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.877648 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:25.877619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:25.896935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:25.896913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:23:25.897042 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:25.897028    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:23:25.897098 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:25.897090    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs podName:a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc nodeName:}" failed. No retries permitted until 2026-09-11 13:25:27.897077271 +0000 UTC m=+252.290764660 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs") pod "network-metrics-daemon-rgwvv" (UID: "a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc") : secret "metrics-daemon-secret" not found
Sep 11 13:23:26.082748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:26.082717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.485396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:26.485362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.681444 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:26.681414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.875772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:26.875744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:26.987732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:26.987707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:27.667837 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:27.667807    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.100643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.100610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.274300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.274270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.333128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.333100    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.386572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.386491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.473775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.473751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.675384 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.675291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:28.878362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:28.878336    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:29.673953 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:29.673922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:30.084647 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:30.084617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:30.472165 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:30.472083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:30.670530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:30.670485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:30.868729 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:30.868701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.260461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:36.260429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.428764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:36.428732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:36.806434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:36.806404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:37.883638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:37.883608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:37.885703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:37.885679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0da8220f-d89e-4f9a-a453-75b2a0947c5d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fds4p\" (UID: \"0da8220f-d89e-4f9a-a453-75b2a0947c5d\") " pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:38.081808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:38.081778    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kd6rn\""
Sep 11 13:23:38.090005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:38.089980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fds4p"
Sep 11 13:23:38.379092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:38.379062    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fds4p"]
Sep 11 13:23:38.383407 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:23:38.383376    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da8220f_d89e_4f9a_a453_75b2a0947c5d.slice/crio-1880d8e58d62f0cc866cbedfb44e09cd338c7f922ae22151006478ffe2cdd893 WatchSource:0}: Error finding container 1880d8e58d62f0cc866cbedfb44e09cd338c7f922ae22151006478ffe2cdd893: Status 404 returned error can't find the container with id 1880d8e58d62f0cc866cbedfb44e09cd338c7f922ae22151006478ffe2cdd893
Sep 11 13:23:38.764999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:38.764971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fds4p" event={"ID":"0da8220f-d89e-4f9a-a453-75b2a0947c5d","Type":"ContainerStarted","Data":"adff5ba989559aa1996315800aed91572cf0b0328970f9e94fd57cf8b97b194e"}
Sep 11 13:23:38.765128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:38.765024    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fds4p" event={"ID":"0da8220f-d89e-4f9a-a453-75b2a0947c5d","Type":"ContainerStarted","Data":"1880d8e58d62f0cc866cbedfb44e09cd338c7f922ae22151006478ffe2cdd893"}
Sep 11 13:23:39.769903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:39.769869    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fds4p" event={"ID":"0da8220f-d89e-4f9a-a453-75b2a0947c5d","Type":"ContainerStarted","Data":"65991bfed8a6f4f66613903210e65a57efdd924fcd701a81315068f695da6f4a"}
Sep 11 13:23:40.775092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:40.775058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fds4p" event={"ID":"0da8220f-d89e-4f9a-a453-75b2a0947c5d","Type":"ContainerStarted","Data":"15fd53c5a4c60b78815f975fa96df0a5ccbae4fcb682989bc9044343dcda8dfd"}
Sep 11 13:23:40.794956 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:40.794919    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fds4p" podStartSLOduration=33.953167715 podStartE2EDuration="35.794905947s" totalImagesPullingTime="1.841738232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:23:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:40.794734349 +0000 UTC m=+145.188421759" watchObservedRunningTime="2026-09-11 13:23:40.794905947 +0000 UTC m=+145.188593358"
Sep 11 13:23:42.877139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:42.877105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:42.966549 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:42.966521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:43.066702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:43.066672    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:43.313504 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:43.313476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:43.369801 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:43.369777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.884736 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:44.884707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:44.985129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:44.985099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.484630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:45.484599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.560586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:45.560552    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.639311 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:45.639279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:45.735633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:45.735555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:46.009455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:46.009380    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:46.109506 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:46.109475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:47.805680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:47.805647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:49.649831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:49.649801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:49.775207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:49.775174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:49.898257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:49.898225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:49.980342 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:49.980259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.011179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.011154    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hlcgl"]
Sep 11 13:23:50.013489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.013468    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.016599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.016000    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pmtd5\""
Sep 11 13:23:50.016599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.016419    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:23:50.016599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.016591    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:23:50.017653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.017009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:23:50.017653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.017257    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:23:50.017653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.017299    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:23:50.017653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.017570    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:23:50.063698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7stq\" (UniqueName: \"kubernetes.io/projected/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-kube-api-access-g7stq\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-textfile\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063782    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-tls\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-wtmp\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063992 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-sys\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063992 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063919    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-root\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.063992 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.063939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.064095 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.064000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-metrics-client-ca\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.065957 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.065937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.159710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.159680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.164728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-metrics-client-ca\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.164792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.164792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7stq\" (UniqueName: \"kubernetes.io/projected/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-kube-api-access-g7stq\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.164867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-textfile\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.164867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-tls\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.164867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-wtmp\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-sys\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-root\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.164948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-wtmp\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-sys\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165129    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-root\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-textfile\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-metrics-client-ca\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.165545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.165529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.167143 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.167126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-tls\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.167194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.167161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.172865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.172843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7stq\" (UniqueName: \"kubernetes.io/projected/4fb00de1-7ea5-4a4e-b917-a2711107b7f2-kube-api-access-g7stq\") pod \"node-exporter-hlcgl\" (UID: \"4fb00de1-7ea5-4a4e-b917-a2711107b7f2\") " pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.259250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.259187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.324666 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.324643    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hlcgl"
Sep 11 13:23:50.334835 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:23:50.334813    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb00de1_7ea5_4a4e_b917_a2711107b7f2.slice/crio-f3d4359eb5850c24d712fa715faf4db2a49fb3061a9dfd3c92f9b9c6e6bf87d8 WatchSource:0}: Error finding container f3d4359eb5850c24d712fa715faf4db2a49fb3061a9dfd3c92f9b9c6e6bf87d8: Status 404 returned error can't find the container with id f3d4359eb5850c24d712fa715faf4db2a49fb3061a9dfd3c92f9b9c6e6bf87d8
Sep 11 13:23:50.364023 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.364001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.804115 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.804077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlcgl" event={"ID":"4fb00de1-7ea5-4a4e-b917-a2711107b7f2","Type":"ContainerStarted","Data":"f3d4359eb5850c24d712fa715faf4db2a49fb3061a9dfd3c92f9b9c6e6bf87d8"}
Sep 11 13:23:50.889395 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.889357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:50.989353 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:50.989320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:51.096399 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:51.096316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:51.808987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:51.808952    2576 generic.go:356] "Generic (PLEG): container finished" podID="4fb00de1-7ea5-4a4e-b917-a2711107b7f2" containerID="16fb7a6b81847ae6763d05e2ac52a01b957249b18b38e0a07b82434524a4415c" exitCode=0
Sep 11 13:23:51.809402 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:51.809020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlcgl" event={"ID":"4fb00de1-7ea5-4a4e-b917-a2711107b7f2","Type":"ContainerDied","Data":"16fb7a6b81847ae6763d05e2ac52a01b957249b18b38e0a07b82434524a4415c"}
Sep 11 13:23:52.814253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:52.814221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlcgl" event={"ID":"4fb00de1-7ea5-4a4e-b917-a2711107b7f2","Type":"ContainerStarted","Data":"314dc8099e676bb7495fc2a7a5da7fdf33355d0095f8e46b216df00d21a59f9c"}
Sep 11 13:23:52.814615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:52.814271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hlcgl" event={"ID":"4fb00de1-7ea5-4a4e-b917-a2711107b7f2","Type":"ContainerStarted","Data":"1eb6200e9946a1417c2d501b557f780628307edc00927de9fed0f100a03fe6c7"}
Sep 11 13:23:52.840247 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:52.840200    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hlcgl" podStartSLOduration=2.966815048 podStartE2EDuration="3.840184476s" totalImagesPullingTime="811.398655ms" totalInitContainerRuntime="61.970773ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:52.838519031 +0000 UTC m=+157.232206434" watchObservedRunningTime="2026-09-11 13:23:52.840184476 +0000 UTC m=+157.233871887"
Sep 11 13:23:53.010455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:53.010420    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:53.100320 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:53.100238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:53.201655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:53.201630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:53.300554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:53.300526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:54.470354 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:54.470303    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" podUID="563ccbae-9f03-4fe5-b582-568955abc290"
Sep 11 13:23:54.533285 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:54.533248    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cn9lc" podUID="5a39d72e-6a80-4205-93a5-8dd0567ba6b1"
Sep 11 13:23:54.694479 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:54.694441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:54.819857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:54.819833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:23:54.819999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:54.819833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:23:55.081396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:55.081308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:56.258672 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:23:56.258638    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rgwvv" podUID="a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc"
Sep 11 13:23:56.417446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:56.417417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:23:59.439655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.439606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:23:59.441973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.441954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"image-registry-5964d6c969-gsk2v\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") " pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:23:59.540263 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.540237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:23:59.542363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.542339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a39d72e-6a80-4205-93a5-8dd0567ba6b1-metrics-tls\") pod \"dns-default-cn9lc\" (UID: \"5a39d72e-6a80-4205-93a5-8dd0567ba6b1\") " pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:23:59.624115 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.624095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vqfbt\""
Sep 11 13:23:59.624186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.624095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zq2ct\""
Sep 11 13:23:59.630957 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.630940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:23:59.631042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.631025    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:23:59.907439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.907418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cn9lc"]
Sep 11 13:23:59.909195 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:23:59.909145    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a39d72e_6a80_4205_93a5_8dd0567ba6b1.slice/crio-d420dc6f6d50296c9110fc56d26762ad707ca36b59b4b4f4eaf6fbeab0eff2c5 WatchSource:0}: Error finding container d420dc6f6d50296c9110fc56d26762ad707ca36b59b4b4f4eaf6fbeab0eff2c5: Status 404 returned error can't find the container with id d420dc6f6d50296c9110fc56d26762ad707ca36b59b4b4f4eaf6fbeab0eff2c5
Sep 11 13:23:59.911023 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:23:59.910983    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:23:59.913205 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:23:59.913179    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563ccbae_9f03_4fe5_b582_568955abc290.slice/crio-6dd17f1f809f4b356bef24df7b2ffb0b768cda5ed1b883a95d0906ff6f266cff WatchSource:0}: Error finding container 6dd17f1f809f4b356bef24df7b2ffb0b768cda5ed1b883a95d0906ff6f266cff: Status 404 returned error can't find the container with id 6dd17f1f809f4b356bef24df7b2ffb0b768cda5ed1b883a95d0906ff6f266cff
Sep 11 13:24:00.839364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:00.839329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" event={"ID":"563ccbae-9f03-4fe5-b582-568955abc290","Type":"ContainerStarted","Data":"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"}
Sep 11 13:24:00.839889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:00.839382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" event={"ID":"563ccbae-9f03-4fe5-b582-568955abc290","Type":"ContainerStarted","Data":"6dd17f1f809f4b356bef24df7b2ffb0b768cda5ed1b883a95d0906ff6f266cff"}
Sep 11 13:24:00.839889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:00.839448    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:24:00.840551 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:00.840524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9lc" event={"ID":"5a39d72e-6a80-4205-93a5-8dd0567ba6b1","Type":"ContainerStarted","Data":"d420dc6f6d50296c9110fc56d26762ad707ca36b59b4b4f4eaf6fbeab0eff2c5"}
Sep 11 13:24:00.862069 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:00.862027    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" podStartSLOduration=162.862012657 podStartE2EDuration="2m42.862012657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:00.860283119 +0000 UTC m=+165.253970529" watchObservedRunningTime="2026-09-11 13:24:00.862012657 +0000 UTC m=+165.255700068"
Sep 11 13:24:01.462668 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.462618    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" podUID="449d1056-0223-44d7-a6d9-fb500f0a1935" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.6:8000/readyz\": dial tcp 10.132.0.6:8000: connect: connection refused"
Sep 11 13:24:01.759239 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.759191    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" podUID="449d1056-0223-44d7-a6d9-fb500f0a1935" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.6:8000/healthz\": dial tcp 10.132.0.6:8000: connect: connection refused"
Sep 11 13:24:01.796789 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.796761    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" podUID="8b5cbd93-22a6-4a03-9813-880f21dcf37d" containerName="addon-agent" probeResult="failure" output="Get \"http://10.132.0.8:8000/healthz\": dial tcp 10.132.0.8:8000: connect: connection refused"
Sep 11 13:24:01.846452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.846364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9lc" event={"ID":"5a39d72e-6a80-4205-93a5-8dd0567ba6b1","Type":"ContainerStarted","Data":"4c6ff0363b8d8c6aa488d7b17bea6fe30d70a078bdd93f8d29e782c50d8b1d99"}
Sep 11 13:24:01.846452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.846431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:24:01.846851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.846457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cn9lc" event={"ID":"5a39d72e-6a80-4205-93a5-8dd0567ba6b1","Type":"ContainerStarted","Data":"4fb9558552d178d82aa7230036ba3c21bd6d73654968191661e6f4ab546335d2"}
Sep 11 13:24:01.847818 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.847796    2576 generic.go:356] "Generic (PLEG): container finished" podID="8b5cbd93-22a6-4a03-9813-880f21dcf37d" containerID="b4500569e2afbe02feab3d40befcb9cd1113f3f303d509257bba59d6fd8f714c" exitCode=255
Sep 11 13:24:01.847912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.847864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" event={"ID":"8b5cbd93-22a6-4a03-9813-880f21dcf37d","Type":"ContainerDied","Data":"b4500569e2afbe02feab3d40befcb9cd1113f3f303d509257bba59d6fd8f714c"}
Sep 11 13:24:01.848168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.848147    2576 scope.go:117] "RemoveContainer" containerID="b4500569e2afbe02feab3d40befcb9cd1113f3f303d509257bba59d6fd8f714c"
Sep 11 13:24:01.849371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.849352    2576 generic.go:356] "Generic (PLEG): container finished" podID="449d1056-0223-44d7-a6d9-fb500f0a1935" containerID="fc254eca1c242f359f54a6c00f3aeb52e9fde4e79599b7352705c14e3faf2ee3" exitCode=1
Sep 11 13:24:01.849446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.849420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" event={"ID":"449d1056-0223-44d7-a6d9-fb500f0a1935","Type":"ContainerDied","Data":"fc254eca1c242f359f54a6c00f3aeb52e9fde4e79599b7352705c14e3faf2ee3"}
Sep 11 13:24:01.849836 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.849818    2576 scope.go:117] "RemoveContainer" containerID="fc254eca1c242f359f54a6c00f3aeb52e9fde4e79599b7352705c14e3faf2ee3"
Sep 11 13:24:01.873807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:01.873768    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cn9lc" podStartSLOduration=129.169676163 podStartE2EDuration="2m10.873755853s" totalImagesPullingTime="1.70407969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:01.873114495 +0000 UTC m=+166.266801907" watchObservedRunningTime="2026-09-11 13:24:01.873755853 +0000 UTC m=+166.267443264"
Sep 11 13:24:02.854942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:02.854903    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c4fcfbf9b-h82d7" event={"ID":"8b5cbd93-22a6-4a03-9813-880f21dcf37d","Type":"ContainerStarted","Data":"03250cc234e7d8ed84929280e4869b4cfa1e22d310983d454c799ff8ce728ecb"}
Sep 11 13:24:02.856778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:02.856752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht" event={"ID":"449d1056-0223-44d7-a6d9-fb500f0a1935","Type":"ContainerStarted","Data":"64bfd9e8415f2fddde8d14fa6b593697fd3cddafabec84cdc581faab75113621"}
Sep 11 13:24:02.857122 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:02.857105    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:24:02.858216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:02.858193    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-68ddbdf896-j78ht"
Sep 11 13:24:05.098258 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.098223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.186180 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.186147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.288157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.288128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.352804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.352725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.561160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.561127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.733534 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.733429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.838191 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.838153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:05.932473 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:05.932445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:06.003098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:06.003011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:06.318932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:06.318902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:06.374705 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:06.374682    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:06.939002 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:06.938970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:07.145599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:07.145565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:07.897377 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:07.897340    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:08.361744 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:08.361711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:08.416115 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:08.416089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:09.242784 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:09.242748    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:24:11.665272 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:11.665226    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9lc_5a39d72e-6a80-4205-93a5-8dd0567ba6b1/dns/0.log"
Sep 11 13:24:11.858878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:11.858846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cn9lc"
Sep 11 13:24:11.865169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:11.865150    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9lc_5a39d72e-6a80-4205-93a5-8dd0567ba6b1/kube-rbac-proxy/0.log"
Sep 11 13:24:12.662034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:12.662007    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tkfh5_ec4d4872-4d83-4d4a-a78f-6945f2447a76/dns-node-resolver/0.log"
Sep 11 13:24:13.065186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:13.065164    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5964d6c969-gsk2v_563ccbae-9f03-4fe5-b582-568955abc290/registry/0.log"
Sep 11 13:24:13.860060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:13.860028    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tq76r_93b4f034-59d0-442e-b948-370d656307d9/node-ca/0.log"
Sep 11 13:24:14.666727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:14.666693    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nllgk_f4b75c12-9d04-448a-9285-b02523dd697e/serve-healthcheck-canary/0.log"
Sep 11 13:24:14.992187 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:14.992102    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:24:14.996229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:14.996194    2576 patch_prober.go:28] interesting pod/image-registry-5964d6c969-gsk2v container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:24:14.996357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:14.996257    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" podUID="563ccbae-9f03-4fe5-b582-568955abc290" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:24:15.148351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:15.148315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:15.245422 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:15.245344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:20.162770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:20.162740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:20.269357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:20.269309    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:21.565780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:21.565747    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:21.618297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:21.618268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:21.807317 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:21.807259    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" podUID="427e42e5-9622-4b3c-b14a-ce64a6f622ed" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 13:24:22.367647 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:22.367609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:22.467484 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:22.467440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:22.526735 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:22.526663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:24.997249 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:24.997221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:24:26.014739 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:26.014704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:26.114320 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:26.114288    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:27.481784 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:27.481753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:27.536203 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:27.536177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.319111 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:28.319078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:28.380405 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:28.380377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:30.012048 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:30.012016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:30.108435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:30.108406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:31.805726 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:31.805691    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" podUID="427e42e5-9622-4b3c-b14a-ce64a6f622ed" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 13:24:35.166766 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:35.166732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:35.276614 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:35.276576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:36.763306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:36.763268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:36.824759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:36.824733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:39.269113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:39.269080    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:40.010776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:40.010730    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" podUID="563ccbae-9f03-4fe5-b582-568955abc290" containerName="registry" containerID="cri-o://ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164" gracePeriod=30
Sep 11 13:24:41.248901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.248864    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:24:41.354338 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354248    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354338 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354294    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354338 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354331    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354349    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354370    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354435    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bj644\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644\") pod \"563ccbae-9f03-4fe5-b582-568955abc290\" (UID: \"563ccbae-9f03-4fe5-b582-568955abc290\") "
Sep 11 13:24:41.354981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:41.354981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.354948    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:41.357204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.357155    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644" (OuterVolumeSpecName: "kube-api-access-bj644") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "kube-api-access-bj644". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:41.357374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.357257    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:41.357374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.357276    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:41.357374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.357288    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:41.357572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.357351    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:41.362633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.362604    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "563ccbae-9f03-4fe5-b582-568955abc290" (UID: "563ccbae-9f03-4fe5-b582-568955abc290"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:24:41.455747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455708    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/563ccbae-9f03-4fe5-b582-568955abc290-ca-trust-extracted\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455740    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-registry-tls\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455750    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-trusted-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455758    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/563ccbae-9f03-4fe5-b582-568955abc290-registry-certificates\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455768    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-bound-sa-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455778    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-image-registry-private-configuration\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455787    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/563ccbae-9f03-4fe5-b582-568955abc290-installation-pull-secrets\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.455987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.455795    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bj644\" (UniqueName: \"kubernetes.io/projected/563ccbae-9f03-4fe5-b582-568955abc290-kube-api-access-bj644\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:24:41.806430 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.806395    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" podUID="427e42e5-9622-4b3c-b14a-ce64a6f622ed" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 11 13:24:41.806602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.806481    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99"
Sep 11 13:24:41.807075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.807042    2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"274b8b70b7c6b90aaaf83ee553d7db0b5aeee8be0c32193b1ac51966ae11c686"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 11 13:24:41.807123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.807099    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" podUID="427e42e5-9622-4b3c-b14a-ce64a6f622ed" containerName="service-proxy" containerID="cri-o://274b8b70b7c6b90aaaf83ee553d7db0b5aeee8be0c32193b1ac51966ae11c686" gracePeriod=30
Sep 11 13:24:41.991019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.990984    2576 generic.go:356] "Generic (PLEG): container finished" podID="563ccbae-9f03-4fe5-b582-568955abc290" containerID="ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164" exitCode=0
Sep 11 13:24:41.991216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.991033    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v"
Sep 11 13:24:41.991216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.991068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" event={"ID":"563ccbae-9f03-4fe5-b582-568955abc290","Type":"ContainerDied","Data":"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"}
Sep 11 13:24:41.991216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.991164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5964d6c969-gsk2v" event={"ID":"563ccbae-9f03-4fe5-b582-568955abc290","Type":"ContainerDied","Data":"6dd17f1f809f4b356bef24df7b2ffb0b768cda5ed1b883a95d0906ff6f266cff"}
Sep 11 13:24:41.991216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.991192    2576 scope.go:117] "RemoveContainer" containerID="ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"
Sep 11 13:24:41.994576 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.994551    2576 generic.go:356] "Generic (PLEG): container finished" podID="427e42e5-9622-4b3c-b14a-ce64a6f622ed" containerID="274b8b70b7c6b90aaaf83ee553d7db0b5aeee8be0c32193b1ac51966ae11c686" exitCode=2
Sep 11 13:24:41.994702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.994622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerDied","Data":"274b8b70b7c6b90aaaf83ee553d7db0b5aeee8be0c32193b1ac51966ae11c686"}
Sep 11 13:24:41.994702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:41.994677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-97768c57c-ntp99" event={"ID":"427e42e5-9622-4b3c-b14a-ce64a6f622ed","Type":"ContainerStarted","Data":"32a13da572ad320962abd5d61f096bb5045bd7078b94a43284438bea6097cdd6"}
Sep 11 13:24:42.000331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:42.000311    2576 scope.go:117] "RemoveContainer" containerID="ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"
Sep 11 13:24:42.000743 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:24:42.000718    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164\": container with ID starting with ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164 not found: ID does not exist" containerID="ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"
Sep 11 13:24:42.000839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:42.000748    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164"} err="failed to get container status \"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164\": rpc error: code = NotFound desc = could not find container \"ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164\": container with ID starting with ed5d048b391b309ecd7860e6733af87dbf4fe5faf4b2ffed0afe38db75811164 not found: ID does not exist"
Sep 11 13:24:42.038110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:42.038077    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:24:42.043039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:42.043014    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5964d6c969-gsk2v"]
Sep 11 13:24:42.247311 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:42.247273    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="563ccbae-9f03-4fe5-b582-568955abc290" path="/var/lib/kubelet/pods/563ccbae-9f03-4fe5-b582-568955abc290/volumes"
Sep 11 13:24:43.315866 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:43.315835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:43.417004 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:43.416956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.196403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:57.196369    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:57.298211 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:57.298179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:24:58.477605 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:24:58.477564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:06.211707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:06.211671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:06.279012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:06.278985    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:21.662420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:21.662388    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:21.722164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:21.722131    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:27.898946 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:27.898899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:25:27.901313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:27.901286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc-metrics-certs\") pod \"network-metrics-daemon-rgwvv\" (UID: \"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc\") " pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:25:28.145803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.145774    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2l67\""
Sep 11 13:25:28.153976 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.153906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rgwvv"
Sep 11 13:25:28.317291 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.317231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:28.417264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.417161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:28.424485 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.424461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rgwvv"]
Sep 11 13:25:28.428089 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:25:28.428048    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d5f6db_e787_4ad7_84d5_a96eee5ee4dc.slice/crio-3e34b719f48f2bccf78e4d0643a635eb5133cad1d769838d401156c9b6de8351 WatchSource:0}: Error finding container 3e34b719f48f2bccf78e4d0643a635eb5133cad1d769838d401156c9b6de8351: Status 404 returned error can't find the container with id 3e34b719f48f2bccf78e4d0643a635eb5133cad1d769838d401156c9b6de8351
Sep 11 13:25:28.518521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:28.518435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:29.147357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:29.147319    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgwvv" event={"ID":"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc","Type":"ContainerStarted","Data":"3e34b719f48f2bccf78e4d0643a635eb5133cad1d769838d401156c9b6de8351"}
Sep 11 13:25:30.153089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:30.153050    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgwvv" event={"ID":"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc","Type":"ContainerStarted","Data":"1e4091c0bbca846ac881277ed2ebfe3935bcb8c2da7a4c1a963433df050e77b9"}
Sep 11 13:25:30.153513 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:30.153107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rgwvv" event={"ID":"a7d5f6db-e787-4ad7-84d5-a96eee5ee4dc","Type":"ContainerStarted","Data":"06477c7ab1aaf8419f29eb9513d2c1723a8e27609f29337a21f385741bc39b3e"}
Sep 11 13:25:30.172641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:30.172487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rgwvv" podStartSLOduration=252.28047158 podStartE2EDuration="4m13.172469806s" totalImagesPullingTime="891.998226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:30.172287256 +0000 UTC m=+254.565974669" watchObservedRunningTime="2026-09-11 13:25:30.172469806 +0000 UTC m=+254.566157220"
Sep 11 13:25:30.302769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:30.302735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:30.366775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:30.366741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:36.219090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:36.219051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:36.284637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:36.284603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:40.890345 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:40.890263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:40.991766 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:40.991735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:46.735286 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:46.735251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:46.835932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:46.835898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:46.896363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:46.896331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:25:53.057097 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:25:53.057059    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:16.005096 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:16.005066    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:26:22.562154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:22.562108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:22.634292 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:22.634262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:49.844300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:49.844262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:50.048712 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:50.048677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:54.073613 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:54.073573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:54.162581 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:54.162551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:55.145961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:55.145925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:55.241283 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:55.241212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:56.004403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:56.004359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:56.100697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:56.100666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:58.017253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.017224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:58.391690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.391609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:26:58.721926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.721850    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"]
Sep 11 13:26:58.722099 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.722067    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="563ccbae-9f03-4fe5-b582-568955abc290" containerName="registry"
Sep 11 13:26:58.722099 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.722078    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="563ccbae-9f03-4fe5-b582-568955abc290" containerName="registry"
Sep 11 13:26:58.722168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.722137    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="563ccbae-9f03-4fe5-b582-568955abc290" containerName="registry"
Sep 11 13:26:58.724731 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.724715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.727072 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.727047    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 13:26:58.727813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.727786    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qf4f9\""
Sep 11 13:26:58.727906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.727813    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 13:26:58.733617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.733590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"]
Sep 11 13:26:58.827088 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.827042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlwkf\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-kube-api-access-zlwkf\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.827088 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.827091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.927526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.927479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlwkf\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-kube-api-access-zlwkf\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.927625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.927541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.936022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.935989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:58.936140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:58.936126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlwkf\" (UniqueName: \"kubernetes.io/projected/63a63c56-1894-4c9f-b315-fba6dca01038-kube-api-access-zlwkf\") pod \"cert-manager-cainjector-559cdd8c87-8r5wx\" (UID: \"63a63c56-1894-4c9f-b315-fba6dca01038\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:59.033923 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:59.033877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"
Sep 11 13:26:59.304625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:59.304469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx"]
Sep 11 13:26:59.307265 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:26:59.307237    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63a63c56_1894_4c9f_b315_fba6dca01038.slice/crio-bcee252ae72c04dde5c9e64e3a9d7e087134edcc10a9ab52392cd573dda24bd1 WatchSource:0}: Error finding container bcee252ae72c04dde5c9e64e3a9d7e087134edcc10a9ab52392cd573dda24bd1: Status 404 returned error can't find the container with id bcee252ae72c04dde5c9e64e3a9d7e087134edcc10a9ab52392cd573dda24bd1
Sep 11 13:26:59.309186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:59.309170    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:26:59.442870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:26:59.442829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx" event={"ID":"63a63c56-1894-4c9f-b315-fba6dca01038","Type":"ContainerStarted","Data":"bcee252ae72c04dde5c9e64e3a9d7e087134edcc10a9ab52392cd573dda24bd1"}
Sep 11 13:27:00.082582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:00.082540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:00.140616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:00.140565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:02.456720 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:02.456635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx" event={"ID":"63a63c56-1894-4c9f-b315-fba6dca01038","Type":"ContainerStarted","Data":"d89a1ff78f1aadf2033fdab070b459f08f3d99cf1b80c233a2c65c51db766604"}
Sep 11 13:27:02.472183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:02.472115    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8r5wx" podStartSLOduration=1.607245665 podStartE2EDuration="4.472092169s" totalImagesPullingTime="2.864846504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:02.472000799 +0000 UTC m=+346.865688209" watchObservedRunningTime="2026-09-11 13:27:02.472092169 +0000 UTC m=+346.865779581"
Sep 11 13:27:03.891848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:03.891819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:03.944639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:03.944609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:06.595555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:06.595519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:09.258407 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:09.258317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:09.359775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:09.359740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:11.163230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:11.163193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:11.265170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:11.265137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:15.602161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:15.602109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:15.709260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:15.709213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:17.619741 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:17.619704    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:29.413438 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:29.413399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:29.512322 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:29.512285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.080181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.080151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.418392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.418310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.508299 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.508268    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.534098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.534072    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"]
Sep 11 13:27:30.537162 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.537148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.540243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.540215    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:27:30.540243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.540241    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:27:30.540388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.540222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:27:30.540388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.540224    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2ksm4\""
Sep 11 13:27:30.540388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.540289    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:27:30.556443 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.556417    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"]
Sep 11 13:27:30.653622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.653590    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79mvk\" (UniqueName: \"kubernetes.io/projected/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-kube-api-access-79mvk\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.653799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.653635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.653799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.653674    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.653799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.653698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.653799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.653720    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.703110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.703039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.754703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.754675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.754847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.754712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79mvk\" (UniqueName: \"kubernetes.io/projected/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-kube-api-access-79mvk\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.754847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.754751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.754847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.754800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.754847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.754839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.755260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.755229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-manifests\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.756972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.756945    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-tmp\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.757206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.757189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.757248 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.757224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-webhook-cert\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.762817 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.762795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79mvk\" (UniqueName: \"kubernetes.io/projected/b2e85ee5-6a8e-47a6-b66c-fa554515c01b-kube-api-access-79mvk\") pod \"opendatahub-operator-controller-manager-9576ccb85-hc9dj\" (UID: \"b2e85ee5-6a8e-47a6-b66c-fa554515c01b\") " pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.801521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.801434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.847032 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.846999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:30.912244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.912162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:30.980405 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:30.980318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.038056 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.038019    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.143481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.143449    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"]
Sep 11 13:27:31.145848 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:27:31.145823    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e85ee5_6a8e_47a6_b66c_fa554515c01b.slice/crio-179225c85d77437026ac1bd51a3d3606feeab5287c1d1dafa61c00c43f302ef0 WatchSource:0}: Error finding container 179225c85d77437026ac1bd51a3d3606feeab5287c1d1dafa61c00c43f302ef0: Status 404 returned error can't find the container with id 179225c85d77437026ac1bd51a3d3606feeab5287c1d1dafa61c00c43f302ef0
Sep 11 13:27:31.177266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.177234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.257713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.257613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.355346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.355316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.456704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.456674    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:31.552092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:31.552051    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj" event={"ID":"b2e85ee5-6a8e-47a6-b66c-fa554515c01b","Type":"ContainerStarted","Data":"179225c85d77437026ac1bd51a3d3606feeab5287c1d1dafa61c00c43f302ef0"}
Sep 11 13:27:34.564688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:34.564652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj" event={"ID":"b2e85ee5-6a8e-47a6-b66c-fa554515c01b","Type":"ContainerStarted","Data":"f601dbb5e358952d53f62825b064584322b554d5619a644de16c688a650cace6"}
Sep 11 13:27:35.569961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:35.569924    2576 generic.go:356] "Generic (PLEG): container finished" podID="b2e85ee5-6a8e-47a6-b66c-fa554515c01b" containerID="f601dbb5e358952d53f62825b064584322b554d5619a644de16c688a650cace6" exitCode=0
Sep 11 13:27:35.570337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:35.570002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj" event={"ID":"b2e85ee5-6a8e-47a6-b66c-fa554515c01b","Type":"ContainerDied","Data":"f601dbb5e358952d53f62825b064584322b554d5619a644de16c688a650cace6"}
Sep 11 13:27:36.575701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:36.575666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj" event={"ID":"b2e85ee5-6a8e-47a6-b66c-fa554515c01b","Type":"ContainerStarted","Data":"747b6c70cbd7e1b91e58eecd8c69720f2b0427669f91db98d5ed9278ff589642"}
Sep 11 13:27:36.576063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:36.575772    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:36.604934 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:36.604875    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj" podStartSLOduration=2.221852795 podStartE2EDuration="6.604852488s" totalImagesPullingTime="3.297039557s" totalInitContainerRuntime="1.085960136s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:36.603017137 +0000 UTC m=+380.996704547" watchObservedRunningTime="2026-09-11 13:27:36.604852488 +0000 UTC m=+380.998539901"
Sep 11 13:27:42.525831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:42.525799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:43.946800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:43.946754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:47.581755 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:47.581723    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9576ccb85-hc9dj"
Sep 11 13:27:49.095789 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.095757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:49.200623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.200597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:49.312948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.312911    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"]
Sep 11 13:27:49.319724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.319704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.328962 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.328940    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 13:27:49.329681 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.329658    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xjglx\""
Sep 11 13:27:49.329681 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.329676    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 13:27:49.329842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.329658    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 13:27:49.329842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.329688    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:27:49.329972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.329958    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 13:27:49.351464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.351414    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"]
Sep 11 13:27:49.396958 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.396918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.397126 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.396967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.397126 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.396993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c44p8\" (UniqueName: \"kubernetes.io/projected/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-kube-api-access-c44p8\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.397126 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.397036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.497801 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.497762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.497801 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.497807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.497995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.497852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c44p8\" (UniqueName: \"kubernetes.io/projected/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-kube-api-access-c44p8\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.497995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.497888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.498464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.498440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-manager-config\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.500417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.500393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-metrics-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.500625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.500604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-cert\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.512708 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.512681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c44p8\" (UniqueName: \"kubernetes.io/projected/74b0bbc3-c829-481c-9c6a-45e53a8dfb87-kube-api-access-c44p8\") pod \"lws-controller-manager-84bdbdc8f4-wm8fb\" (UID: \"74b0bbc3-c829-481c-9c6a-45e53a8dfb87\") " pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.628346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.628274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:49.784003 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.783966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:49.933598 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.933515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:49.949073 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:49.949046    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"]
Sep 11 13:27:49.952203 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:27:49.952179    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b0bbc3_c829_481c_9c6a_45e53a8dfb87.slice/crio-917f3c49630edb10c86b9552358c874b84a0e296a33285d6113d06cd5b72bbf6 WatchSource:0}: Error finding container 917f3c49630edb10c86b9552358c874b84a0e296a33285d6113d06cd5b72bbf6: Status 404 returned error can't find the container with id 917f3c49630edb10c86b9552358c874b84a0e296a33285d6113d06cd5b72bbf6
Sep 11 13:27:50.305561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:50.305525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:50.589118 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:50.589029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:50.624055 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:50.623978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb" event={"ID":"74b0bbc3-c829-481c-9c6a-45e53a8dfb87","Type":"ContainerStarted","Data":"917f3c49630edb10c86b9552358c874b84a0e296a33285d6113d06cd5b72bbf6"}
Sep 11 13:27:51.782756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:51.782722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:51.854449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:51.854411    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:52.632897 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:52.632862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb" event={"ID":"74b0bbc3-c829-481c-9c6a-45e53a8dfb87","Type":"ContainerStarted","Data":"771abfca340524de1ef5e4587322fc9c0ec0a485f90ab36949e21971e633789e"}
Sep 11 13:27:52.633130 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:52.632960    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:27:52.650585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:52.650535    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb" podStartSLOduration=1.544202252 podStartE2EDuration="3.650518775s" totalImagesPullingTime="2.106316523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:27:52.648721619 +0000 UTC m=+397.042409030" watchObservedRunningTime="2026-09-11 13:27:52.650518775 +0000 UTC m=+397.044206183"
Sep 11 13:27:52.976591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:52.976515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:57.100266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:57.100232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:57.199638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:57.199609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:58.137323 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.137290    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"]
Sep 11 13:27:58.140654 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.140635    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.143100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.143074    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:27:58.143220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.143126    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:27:58.143369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.143353    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:27:58.143808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.143790    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gmphr\""
Sep 11 13:27:58.156370 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.156344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"]
Sep 11 13:27:58.159631 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.159587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:58.161635 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161614    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161714 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161714 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gjs4\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-kube-api-access-8gjs4\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161939 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.161939 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161918    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.162003 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.161961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.262675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262642    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.262675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.262912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262693    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gjs4\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-kube-api-access-8gjs4\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.262912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.262912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262877    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.262961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263444 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263614 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.263684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.263656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.265135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.265113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.265357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.265339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.271541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.271513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gjs4\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-kube-api-access-8gjs4\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.271815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.271798    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/baacab4d-9cb0-46af-bc3f-a65f7ce4d74b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6\" (UID: \"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.275536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.275513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:58.450595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.450515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:27:58.559143 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.559112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:27:58.735971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:58.735885    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"]
Sep 11 13:27:58.738728 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:27:58.738700    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaacab4d_9cb0_46af_bc3f_a65f7ce4d74b.slice/crio-f4446c8494c38ce6451a6b31932aecaeba6184884a5953408d11ef03c0bb65b0 WatchSource:0}: Error finding container f4446c8494c38ce6451a6b31932aecaeba6184884a5953408d11ef03c0bb65b0: Status 404 returned error can't find the container with id f4446c8494c38ce6451a6b31932aecaeba6184884a5953408d11ef03c0bb65b0
Sep 11 13:27:59.657080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:27:59.657046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6" event={"ID":"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b","Type":"ContainerStarted","Data":"f4446c8494c38ce6451a6b31932aecaeba6184884a5953408d11ef03c0bb65b0"}
Sep 11 13:28:01.247742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.247691    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kv2sr"]
Sep 11 13:28:01.254703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.254678    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.262931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.262844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 13:28:01.263910 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.263887    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-q2tdc\""
Sep 11 13:28:01.265848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.265821    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kv2sr"]
Sep 11 13:28:01.288929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.288887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.288929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.288928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxqzd\" (UniqueName: \"kubernetes.io/projected/4165f24d-343b-4bb9-8dd9-354b7e639e71-kube-api-access-jxqzd\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.327904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.327868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:01.390149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.390113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.390149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.390157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxqzd\" (UniqueName: \"kubernetes.io/projected/4165f24d-343b-4bb9-8dd9-354b7e639e71-kube-api-access-jxqzd\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.390370 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:28:01.390284    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 13:28:01.390370 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:28:01.390361    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert podName:4165f24d-343b-4bb9-8dd9-354b7e639e71 nodeName:}" failed. No retries permitted until 2026-09-11 13:28:01.890340123 +0000 UTC m=+406.284027511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert") pod "odh-model-controller-7998c67dc4-kv2sr" (UID: "4165f24d-343b-4bb9-8dd9-354b7e639e71") : secret "odh-model-controller-webhook-cert" not found
Sep 11 13:28:01.411089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.411052    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxqzd\" (UniqueName: \"kubernetes.io/projected/4165f24d-343b-4bb9-8dd9-354b7e639e71-kube-api-access-jxqzd\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.466446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.466414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:01.594741 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.594708    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:01.679399 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.679362    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:28:01.679505 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.679443    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:28:01.679505 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.679473    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:28:01.699138 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.699103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:01.895006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.894908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:01.897480 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:01.897444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4165f24d-343b-4bb9-8dd9-354b7e639e71-cert\") pod \"odh-model-controller-7998c67dc4-kv2sr\" (UID: \"4165f24d-343b-4bb9-8dd9-354b7e639e71\") " pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:02.131246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.131203    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v4p74"]
Sep 11 13:28:02.134489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.134462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.140719 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.140692    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 13:28:02.154146 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.154062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:02.156645 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.156613    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v4p74"]
Sep 11 13:28:02.167174 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.167143    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:02.197600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.197565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pckqr\" (UniqueName: \"kubernetes.io/projected/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-kube-api-access-pckqr\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.197771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.197627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.298594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.298552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.298969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.298649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pckqr\" (UniqueName: \"kubernetes.io/projected/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-kube-api-access-pckqr\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.298969 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:28:02.298710    2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 13:28:02.298969 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:28:02.298783    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs podName:6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa nodeName:}" failed. No retries permitted until 2026-09-11 13:28:02.798763191 +0000 UTC m=+407.192450596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs") pod "model-serving-api-54d89bd5fc-v4p74" (UID: "6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa") : secret "model-serving-api-tls" not found
Sep 11 13:28:02.319160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.319128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pckqr\" (UniqueName: \"kubernetes.io/projected/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-kube-api-access-pckqr\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.456725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.456651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kv2sr"]
Sep 11 13:28:02.459749 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:28:02.459717    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4165f24d_343b_4bb9_8dd9_354b7e639e71.slice/crio-513b2af930d1011ded64a6bd1b5ec3104fb7e579c9414499e1074ab9b13238ba WatchSource:0}: Error finding container 513b2af930d1011ded64a6bd1b5ec3104fb7e579c9414499e1074ab9b13238ba: Status 404 returned error can't find the container with id 513b2af930d1011ded64a6bd1b5ec3104fb7e579c9414499e1074ab9b13238ba
Sep 11 13:28:02.610235 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.610192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:02.671308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.671273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr" event={"ID":"4165f24d-343b-4bb9-8dd9-354b7e639e71","Type":"ContainerStarted","Data":"513b2af930d1011ded64a6bd1b5ec3104fb7e579c9414499e1074ab9b13238ba"}
Sep 11 13:28:02.672790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.672763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6" event={"ID":"baacab4d-9cb0-46af-bc3f-a65f7ce4d74b","Type":"ContainerStarted","Data":"77389171452af662535d559c83e82a9060a3366b0b9bbdad121b87cebc6c8205"}
Sep 11 13:28:02.709287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.709167    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6" podStartSLOduration=1.7729569870000002 podStartE2EDuration="4.70914403s" totalImagesPullingTime="2.936187043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:28:02.708098311 +0000 UTC m=+407.101785721" watchObservedRunningTime="2026-09-11 13:28:02.70914403 +0000 UTC m=+407.102831442"
Sep 11 13:28:02.802634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.802589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:02.805087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:02.805054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa-tls-certs\") pod \"model-serving-api-54d89bd5fc-v4p74\" (UID: \"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa\") " pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:03.045781 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.045485    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:03.346975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.346838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-v4p74"]
Sep 11 13:28:03.351724 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:28:03.351691    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b43c580_f9a2_4dd4_9f5a_8c9e8336ccaa.slice/crio-441fc06eba7670a619ba1ef7614c3d450243010eaae27418af88478966a36e85 WatchSource:0}: Error finding container 441fc06eba7670a619ba1ef7614c3d450243010eaae27418af88478966a36e85: Status 404 returned error can't find the container with id 441fc06eba7670a619ba1ef7614c3d450243010eaae27418af88478966a36e85
Sep 11 13:28:03.451562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.451524    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:28:03.457894 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.457829    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:28:03.639616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.639460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84bdbdc8f4-wm8fb"
Sep 11 13:28:03.686079 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.686038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74" event={"ID":"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa","Type":"ContainerStarted","Data":"441fc06eba7670a619ba1ef7614c3d450243010eaae27418af88478966a36e85"}
Sep 11 13:28:03.686436 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.686408    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:28:03.687528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:03.687476    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6"
Sep 11 13:28:05.696594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:05.696555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr" event={"ID":"4165f24d-343b-4bb9-8dd9-354b7e639e71","Type":"ContainerStarted","Data":"483b5c166161faabac4006e3e33fa932eb311b88b2ad08cc0babbaf4a76fe208"}
Sep 11 13:28:05.697062 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:05.696659    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:07.707059 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:07.707027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74" event={"ID":"6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa","Type":"ContainerStarted","Data":"61d7c5b4bbf321bdf98294bd80ef3399523d239cfcd8ba2281d418cb8121c9bb"}
Sep 11 13:28:07.707465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:07.707269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:28:07.727389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:07.727322    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr" podStartSLOduration=4.195853592 podStartE2EDuration="6.727301763s" totalImagesPullingTime="2.531448171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:28:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:28:05.715453735 +0000 UTC m=+410.109141147" watchObservedRunningTime="2026-09-11 13:28:07.727301763 +0000 UTC m=+412.120989175"
Sep 11 13:28:07.729351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:07.729302    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74" podStartSLOduration=2.114599513 podStartE2EDuration="5.729287297s" totalImagesPullingTime="3.614687784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:28:07.727242363 +0000 UTC m=+412.120929776" watchObservedRunningTime="2026-09-11 13:28:07.729287297 +0000 UTC m=+412.122974706"
Sep 11 13:28:08.994165 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:08.994133    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"]
Sep 11 13:28:08.997023 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:08.997006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.025088 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.025063    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 13:28:09.026206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.026182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 13:28:09.026281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.026185    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 13:28:09.035936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.035914    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c5xh9\""
Sep 11 13:28:09.056093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmzdn\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-kube-api-access-cmzdn\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6491d9c-b73d-481a-8c82-f16de3694df4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056205    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056249    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.056309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.056271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.125957 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.125927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"]
Sep 11 13:28:09.156928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.156894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmzdn\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-kube-api-access-cmzdn\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.156928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.156928    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6491d9c-b73d-481a-8c82-f16de3694df4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.156957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.156973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.157088    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.157142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157354 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.157170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.157909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.157880    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.159466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.159444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.159572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.159535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6491d9c-b73d-481a-8c82-f16de3694df4-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.159572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.159540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.159673 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.159657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.179226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.179205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.180956 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.180939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmzdn\" (UniqueName: \"kubernetes.io/projected/e6491d9c-b73d-481a-8c82-f16de3694df4-kube-api-access-cmzdn\") pod \"istiod-openshift-gateway-5677ccb49b-shkfw\" (UID: \"e6491d9c-b73d-481a-8c82-f16de3694df4\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.306221 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.306187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:09.587571 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.587547    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"]
Sep 11 13:28:09.589481 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:28:09.589455    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6491d9c_b73d_481a_8c82_f16de3694df4.slice/crio-fa978032362b1a1fa182c0af4e1ef846198222c577045740f7d1cb40335aa461 WatchSource:0}: Error finding container fa978032362b1a1fa182c0af4e1ef846198222c577045740f7d1cb40335aa461: Status 404 returned error can't find the container with id fa978032362b1a1fa182c0af4e1ef846198222c577045740f7d1cb40335aa461
Sep 11 13:28:09.717702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:09.717668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw" event={"ID":"e6491d9c-b73d-481a-8c82-f16de3694df4","Type":"ContainerStarted","Data":"fa978032362b1a1fa182c0af4e1ef846198222c577045740f7d1cb40335aa461"}
Sep 11 13:28:11.644466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:11.644426    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:28:11.644761 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:11.644517    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:28:11.729427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:11.729376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw" event={"ID":"e6491d9c-b73d-481a-8c82-f16de3694df4","Type":"ContainerStarted","Data":"302397094e126aa0695f75a64887e111d7874fe113704f5cc802c7d122dd6375"}
Sep 11 13:28:11.729644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:11.729484    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:11.777895 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:11.777837    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw" podStartSLOduration=1.725099744 podStartE2EDuration="3.777819798s" totalImagesPullingTime="2.052720054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:28:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:28:11.777374968 +0000 UTC m=+416.171062380" watchObservedRunningTime="2026-09-11 13:28:11.777819798 +0000 UTC m=+416.171507209"
Sep 11 13:28:12.735008 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:12.734978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-shkfw"
Sep 11 13:28:12.961873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:12.961781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:13.111593 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:13.111560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:13.740458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:13.740426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:13.823028 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:13.822996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:15.695311 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:15.695273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:15.794116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:15.794072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:15.901661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:15.901623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:28:16.704048 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:16.704020    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kv2sr"
Sep 11 13:28:18.715116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:28:18.715084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-v4p74"
Sep 11 13:29:01.578769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:01.578733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:01.640942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:01.640906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:01.935594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:01.935490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:02.028537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:02.028475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:16.105318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:16.105282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:16.908919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:16.908888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:19.633826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:19.633796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:19.737307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:19.737261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:20.328307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:20.328271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:20.722858 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:20.722773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:21.220383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:21.220350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:21.352805 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:21.352771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:21.727001 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:21.726965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:21.844147 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:21.844113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:24.734912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:24.734877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:26.184731 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:26.184698    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:26.928636 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:26.928601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:27.673426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:27.673391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:28.471438 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:28.471404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:28.579364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:28.579335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:30.188702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:30.188666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:30.295958 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:30.295919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:30.388757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:30.388726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:30.488842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:30.488736    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:32.126622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:32.126585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:32.234701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:32.234657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:32.354264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:32.354228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:32.696293 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:32.696261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:32.888133 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:32.888098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:36.002082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:36.002052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:36.099574 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:36.099544    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:36.205768 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:36.205741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:36.307263 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:36.307233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:42.996639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:42.996597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:43.126142 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:43.126112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:47.340666 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:47.340631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:47.887622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:47.887571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:50.285829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:50.285788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:50.388073 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:50.388041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:50.486792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:50.486756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:55.846520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:55.846450    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:55.946113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:55.946078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:56.254364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:56.254259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:56.358411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:56.358377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:56.628852 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:56.628821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:29:56.726860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:29:56.726822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:02.054457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:02.054422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:02.215051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:02.215010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:05.054815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:05.054780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:05.105841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:05.105793    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:06.784607 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:06.784567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:06.885638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:06.885603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:13.331361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:13.331239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:13.697696 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:13.697593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:27.883765 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:27.883726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:28.278424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:28.278394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:28.378869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:28.378833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:28.479860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:28.479832    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:29.145451 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:29.145416    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:29.229197 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:29.229153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:59.337682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:59.337650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:59.439155 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:59.439119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:30:59.800786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:30:59.800755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:08.929247 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:08.929203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:09.760049 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:09.760012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:12.664351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:12.664312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:13.226239 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:13.226207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:19.973656 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:19.973620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.072615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:20.072581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.127823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:20.127784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.250074 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:20.249992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:20.308934 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:20.308906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:23.690763 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:23.690730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:23.790304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:23.790266    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:55.303827 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:55.303790    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:55.393110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:55.393076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:55.493874 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:55.493846    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:56.853102 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:56.853070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:56.969639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:56.969609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.075640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:57.075611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.233626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:57.233543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.541308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:57.541276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:31:57.641404 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:31:57.641373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:00.593809 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:00.593769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:00.647409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:00.647376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:08.692829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:08.692791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:08.766921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:08.766882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:17.339424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:17.339390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:17.440156 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:17.440127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:20.562609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.562576    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"]
Sep 11 13:32:20.565815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.565794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.568454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.568436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:32:20.569140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.569125    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:32:20.569205 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.569129    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 13:32:20.569409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.569393    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6z2kw\""
Sep 11 13:32:20.575025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.575003    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"]
Sep 11 13:32:20.575539 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.575514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:20.736042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8q5g\" (UniqueName: \"kubernetes.io/projected/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-kube-api-access-v8q5g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.736042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.736287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736083    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.736287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.736287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.736287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.736271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8q5g\" (UniqueName: \"kubernetes.io/projected/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-kube-api-access-v8q5g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.837855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.837748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.839467 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.839447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.839670 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.839653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.850165 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.850142    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8q5g\" (UniqueName: \"kubernetes.io/projected/d41a4e3b-8a4c-4722-ae70-0da2325a2b29-kube-api-access-v8q5g\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq\" (UID: \"d41a4e3b-8a4c-4722-ae70-0da2325a2b29\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:20.876324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:20.876288    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:21.150331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:21.150307    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"]
Sep 11 13:32:21.152780 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:32:21.152753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41a4e3b_8a4c_4722_ae70_0da2325a2b29.slice/crio-6a76f26c20ae4c805de5483f90beedeb26f33725d0fd2839dbc4db92f72d9800 WatchSource:0}: Error finding container 6a76f26c20ae4c805de5483f90beedeb26f33725d0fd2839dbc4db92f72d9800: Status 404 returned error can't find the container with id 6a76f26c20ae4c805de5483f90beedeb26f33725d0fd2839dbc4db92f72d9800
Sep 11 13:32:21.154098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:21.154082    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:32:21.646971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:21.646932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq" event={"ID":"d41a4e3b-8a4c-4722-ae70-0da2325a2b29","Type":"ContainerStarted","Data":"6a76f26c20ae4c805de5483f90beedeb26f33725d0fd2839dbc4db92f72d9800"}
Sep 11 13:32:22.652379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:22.652343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq" event={"ID":"d41a4e3b-8a4c-4722-ae70-0da2325a2b29","Type":"ContainerStarted","Data":"afe123981be498622f3dba193ce736a6b073cf2aa29b2e21960e676521377058"}
Sep 11 13:32:22.673374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:22.673313    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq" podStartSLOduration=1.68554578 podStartE2EDuration="2.673289833s" totalImagesPullingTime="987.744053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:32:22.671699769 +0000 UTC m=+667.065387179" watchObservedRunningTime="2026-09-11 13:32:22.673289833 +0000 UTC m=+667.066977246"
Sep 11 13:32:28.248227 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.248189    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"]
Sep 11 13:32:28.251486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.251464    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.254078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.254061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 11 13:32:28.261374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.261348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"]
Sep 11 13:32:28.269640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.269618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:28.296437 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.296585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wppn\" (UniqueName: \"kubernetes.io/projected/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-kube-api-access-8wppn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.296585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.296585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.296684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.296684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.296615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wppn\" (UniqueName: \"kubernetes.io/projected/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-kube-api-access-8wppn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397527 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.397644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.397608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.399436 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.399411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.399625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.399607    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.410723 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.410699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wppn\" (UniqueName: \"kubernetes.io/projected/e8c42d78-84ad-469d-9187-5b7f70f5e6fe-kube-api-access-8wppn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f7w6x\" (UID: \"e8c42d78-84ad-469d-9187-5b7f70f5e6fe\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.563267 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.563231    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:28.836046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:28.835969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"]
Sep 11 13:32:28.838303 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:32:28.838279    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c42d78_84ad_469d_9187_5b7f70f5e6fe.slice/crio-a479bc82cff96b7786d2daf4885b9e6b1b436cb427ae076c507a47dc00a58415 WatchSource:0}: Error finding container a479bc82cff96b7786d2daf4885b9e6b1b436cb427ae076c507a47dc00a58415: Status 404 returned error can't find the container with id a479bc82cff96b7786d2daf4885b9e6b1b436cb427ae076c507a47dc00a58415
Sep 11 13:32:29.681215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:29.681179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x" event={"ID":"e8c42d78-84ad-469d-9187-5b7f70f5e6fe","Type":"ContainerStarted","Data":"4acba466d5851188a021e028bdd0bb44f0ff2e059b257ff8b1c94f7024db6912"}
Sep 11 13:32:29.681661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:29.681235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x" event={"ID":"e8c42d78-84ad-469d-9187-5b7f70f5e6fe","Type":"ContainerStarted","Data":"a479bc82cff96b7786d2daf4885b9e6b1b436cb427ae076c507a47dc00a58415"}
Sep 11 13:32:29.704011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:29.703957    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x" podStartSLOduration=1.515158555 podStartE2EDuration="1.70393841s" totalImagesPullingTime="188.779855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:32:29.702753393 +0000 UTC m=+674.096440805" watchObservedRunningTime="2026-09-11 13:32:29.70393841 +0000 UTC m=+674.097625820"
Sep 11 13:32:30.682417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:30.682378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:30.782712 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:30.782679    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:30.876560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:30.876489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:30.876764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:30.876589    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:30.889005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:30.888970    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:31.699867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:31.699838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq"
Sep 11 13:32:35.467451 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:35.467424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:38.564082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:38.564051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:38.564468 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:38.564106    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:38.576414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:38.576389    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:38.726643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:38.726614    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f7w6x"
Sep 11 13:32:40.244033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:40.243998    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:56.866889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.866852    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"]
Sep 11 13:32:56.871687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.871668    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.873730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.873701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:32:56.874828 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.874808    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 13:32:56.883948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.883925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"]
Sep 11 13:32:56.924440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.924617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e6a84f-fe78-4af0-8526-7e16d41ca986-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.924617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.924617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924602    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.924746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:56.924746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:56.924666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gr6h\" (UniqueName: \"kubernetes.io/projected/65e6a84f-fe78-4af0-8526-7e16d41ca986-kube-api-access-6gr6h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026176    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e6a84f-fe78-4af0-8526-7e16d41ca986-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gr6h\" (UniqueName: \"kubernetes.io/projected/65e6a84f-fe78-4af0-8526-7e16d41ca986-kube-api-access-6gr6h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026740 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026740 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.026986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.026925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.028587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.028564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65e6a84f-fe78-4af0-8526-7e16d41ca986-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.028828 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.028811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65e6a84f-fe78-4af0-8526-7e16d41ca986-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.035580 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.035552    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gr6h\" (UniqueName: \"kubernetes.io/projected/65e6a84f-fe78-4af0-8526-7e16d41ca986-kube-api-access-6gr6h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm\" (UID: \"65e6a84f-fe78-4af0-8526-7e16d41ca986\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.182394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.182302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:32:57.457440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.457323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"]
Sep 11 13:32:57.460001 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:32:57.459972    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e6a84f_fe78_4af0_8526_7e16d41ca986.slice/crio-ce8b27ce6a920e656e9996053219cf771914d53754d2671eed2e42deadf40dc2 WatchSource:0}: Error finding container ce8b27ce6a920e656e9996053219cf771914d53754d2671eed2e42deadf40dc2: Status 404 returned error can't find the container with id ce8b27ce6a920e656e9996053219cf771914d53754d2671eed2e42deadf40dc2
Sep 11 13:32:57.794975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.794936    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm" event={"ID":"65e6a84f-fe78-4af0-8526-7e16d41ca986","Type":"ContainerStarted","Data":"a9c76d76da34b95329978fd5ae15a27c8f06a465873fa390e2739f71c50186d2"}
Sep 11 13:32:57.795150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.794988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm" event={"ID":"65e6a84f-fe78-4af0-8526-7e16d41ca986","Type":"ContainerStarted","Data":"ce8b27ce6a920e656e9996053219cf771914d53754d2671eed2e42deadf40dc2"}
Sep 11 13:32:57.814428 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:32:57.814372    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm" podStartSLOduration=1.597264601 podStartE2EDuration="1.814356307s" totalImagesPullingTime="217.091706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:32:57.813167658 +0000 UTC m=+702.206855071" watchObservedRunningTime="2026-09-11 13:32:57.814356307 +0000 UTC m=+702.208043780"
Sep 11 13:33:05.863536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:05.863485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:33:07.183228 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:07.183186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:33:07.183709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:07.183251    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:33:07.195975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:07.195946    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:33:07.846035 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:07.846002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm"
Sep 11 13:33:09.088867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:33:09.088835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:22.662829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:22.662789    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:32.606829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:32.606786    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:32.714862 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:32.714825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:52.898131 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:52.898088    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:52.995246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:52.995213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:54.067926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:54.067897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:34:54.169667 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:34:54.169635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:10.943548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:10.943487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:11.059271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:11.059234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:11.451876 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:11.451840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:11.845358 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:11.845327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:12.238374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.238293    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:12.285700 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.285666    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:35:12.289062 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.289030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.293837 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.293810    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-b65337-openshift-default-dockercfg-pbx2x\""
Sep 11 13:35:12.323375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323342    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbth\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323674    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.323843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.323691    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.333129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.333100    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:35:12.370285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.370250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:12.424856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbth\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.424856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424913    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.424983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425354 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425354 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.425926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.425905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.427356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.427338    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.427412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.427370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.490737 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.490646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbth\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.492194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.492165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token\") pod \"e2e-inf-a-b65337-openshift-default-6765579894-9c4vh\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") " pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.599730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.599694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:12.671092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.671061    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:12.804216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.804180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:12.884948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.884918    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:35:12.887301 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:12.887271    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59523553_7bcf_4f40_b6f1_84377f8d2d89.slice/crio-1afbf201a56b7e62061bf9b514f1efbeb81d7a46f88437355dabaf2fc4e40d57 WatchSource:0}: Error finding container 1afbf201a56b7e62061bf9b514f1efbeb81d7a46f88437355dabaf2fc4e40d57: Status 404 returned error can't find the container with id 1afbf201a56b7e62061bf9b514f1efbeb81d7a46f88437355dabaf2fc4e40d57
Sep 11 13:35:12.889244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.889215    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:12.889297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.889274    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:12.889330 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.889303    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:12.982427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:12.982397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:13.073590 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.073483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:13.173443 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.173413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:13.336420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.336330    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerStarted","Data":"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"}
Sep 11 13:35:13.336420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.336380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerStarted","Data":"1afbf201a56b7e62061bf9b514f1efbeb81d7a46f88437355dabaf2fc4e40d57"}
Sep 11 13:35:13.365247 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.365198    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podStartSLOduration=1.365182415 podStartE2EDuration="1.365182415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:13.363977914 +0000 UTC m=+837.757665325" watchObservedRunningTime="2026-09-11 13:35:13.365182415 +0000 UTC m=+837.758869865"
Sep 11 13:35:13.424305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.424272    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:13.600173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.600073    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:13.605198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:13.605173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:14.340806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:14.340775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:14.341694 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:14.341675    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:15.038904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:15.038867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:15.181506 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:15.181474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:15.280886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:15.280853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:15.592460 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:15.592425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:15.740246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:15.740198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:16.028936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:16.028888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:16.110786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:16.110753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:17.191768 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:17.191722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:17.328873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:17.328840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:17.819044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:17.819011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:18.576023 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.575990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:18.686560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.686486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:18.847873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.847792    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:35:18.851066 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.851050    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:18.853821 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.853797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-b65337\""
Sep 11 13:35:18.863914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.863894    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:35:18.866911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.866889    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:35:18.867026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.866972    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:18.872909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.872868    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-b65337-dockercfg-q4jk5\""
Sep 11 13:35:18.887289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.887259    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:35:18.983041 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.983006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxrn\" (UniqueName: \"kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:18.983209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.983064    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9rv\" (UniqueName: \"kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:18.983209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.983099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:18.983209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:18.983117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.084404 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.084364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxrn\" (UniqueName: \"kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.084632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.084428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9rv\" (UniqueName: \"kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:19.084632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.084462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:19.084632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.084480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.085212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.085191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.085212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.085204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:19.096135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.096109    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9rv\" (UniqueName: \"kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv\") pod \"payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:19.096414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.096395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxrn\" (UniqueName: \"kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn\") pod \"payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") " pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.161366 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.161281    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:19.176591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.176561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:19.335686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.335639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:19.454091 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.454011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:19.466075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.466050    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:35:19.468808 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:19.468780    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b7c0e2_0d13_44c8_89b6_2d491078d451.slice/crio-742593dbd8663e8cdb988ef4c76a2bf4683e0b59b8d112ddfc45d2d8d0e52866 WatchSource:0}: Error finding container 742593dbd8663e8cdb988ef4c76a2bf4683e0b59b8d112ddfc45d2d8d0e52866: Status 404 returned error can't find the container with id 742593dbd8663e8cdb988ef4c76a2bf4683e0b59b8d112ddfc45d2d8d0e52866
Sep 11 13:35:19.477144 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:19.477065    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:35:19.479571 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:19.479542    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52023f0_47cb_4a69_98bd_eb8c89c6f5c9.slice/crio-6cc4bc3075ac55311e64f86e64af26eecf75d8405b8f403538cb55e50ae24d63 WatchSource:0}: Error finding container 6cc4bc3075ac55311e64f86e64af26eecf75d8405b8f403538cb55e50ae24d63: Status 404 returned error can't find the container with id 6cc4bc3075ac55311e64f86e64af26eecf75d8405b8f403538cb55e50ae24d63
Sep 11 13:35:20.171057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.171021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:20.277162 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.277115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:20.374897 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.374813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" event={"ID":"e6b7c0e2-0d13-44c8-89b6-2d491078d451","Type":"ContainerStarted","Data":"742593dbd8663e8cdb988ef4c76a2bf4683e0b59b8d112ddfc45d2d8d0e52866"}
Sep 11 13:35:20.376488 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.376373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" event={"ID":"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9","Type":"ContainerStarted","Data":"6cc4bc3075ac55311e64f86e64af26eecf75d8405b8f403538cb55e50ae24d63"}
Sep 11 13:35:20.388184 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.388121    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:20.939398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:20.939364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.048333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.048286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.192232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.192150    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.428011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.427976    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:35:21.431466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.431440    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.435380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.435352    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-f9f26e-openshift-default-dockercfg-cr42w\""
Sep 11 13:35:21.461435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.461322    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:35:21.462007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.461974    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.509278 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q94rh\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509278 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509290    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509558    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.509769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.509736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610250    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q94rh\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610434    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610834 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610834 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610850    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.610988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.610935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.611076 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.611049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.611219 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.611158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.612877 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.612834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.613297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.613276    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.634360 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.634296    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q94rh\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.636018 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.635983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token\") pod \"e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") " pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.648491 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.648449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:21.742572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.742424    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:21.811551 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:21.811519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:22.047096 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.047068    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:35:22.049299 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:22.049266    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f269c6_c533_4037_a224_8e2b2afb903a.slice/crio-1a81f6cf238f0537c1e7fd15835d391ab52125ce0f4f24feb944bb0d7076b016 WatchSource:0}: Error finding container 1a81f6cf238f0537c1e7fd15835d391ab52125ce0f4f24feb944bb0d7076b016: Status 404 returned error can't find the container with id 1a81f6cf238f0537c1e7fd15835d391ab52125ce0f4f24feb944bb0d7076b016
Sep 11 13:35:22.051880 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.051848    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:22.051948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.051912    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:22.051948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.051939    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:22.388913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.388828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerStarted","Data":"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"}
Sep 11 13:35:22.388913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.388883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerStarted","Data":"1a81f6cf238f0537c1e7fd15835d391ab52125ce0f4f24feb944bb0d7076b016"}
Sep 11 13:35:22.390402 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.390378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" event={"ID":"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9","Type":"ContainerStarted","Data":"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"}
Sep 11 13:35:22.390524 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.390431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:22.391921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.391902    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" event={"ID":"e6b7c0e2-0d13-44c8-89b6-2d491078d451","Type":"ContainerStarted","Data":"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"}
Sep 11 13:35:22.392051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.392037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:22.438916 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.438849    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podStartSLOduration=1.438829324 podStartE2EDuration="1.438829324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:22.437608896 +0000 UTC m=+846.831296306" watchObservedRunningTime="2026-09-11 13:35:22.438829324 +0000 UTC m=+846.832516733"
Sep 11 13:35:22.449539 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.449490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:22.494567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.494487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" podStartSLOduration=2.63971051 podStartE2EDuration="4.494439413s" totalImagesPullingTime="1.854728903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:22.493164245 +0000 UTC m=+846.886851657" watchObservedRunningTime="2026-09-11 13:35:22.494439413 +0000 UTC m=+846.888126825"
Sep 11 13:35:22.547185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.547120    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" podStartSLOduration=2.69621915 podStartE2EDuration="4.547098585s" totalImagesPullingTime="1.850879435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:22.541047753 +0000 UTC m=+846.934735146" watchObservedRunningTime="2026-09-11 13:35:22.547098585 +0000 UTC m=+846.940785997"
Sep 11 13:35:22.573893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.573852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:22.742657 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:22.742566    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:23.517600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:23.517568    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:23.748435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:23.748400    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:24.401082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:24.401056    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:24.402045 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:24.402026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:24.651200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:24.651098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.767738 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:24.767701    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:24.862159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:24.862124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:25.344558 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:25.344524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:25.406890 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:25.406859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:25.642776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:25.642696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:25.936925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:25.936834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:26.018562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.018527    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:35:26.030313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.030279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.038433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.038377    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ae1d6b61-openshift-default-dockercfg-464zd\""
Sep 11 13:35:26.041313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.041282    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:35:26.046337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.046313    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:26.152650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.152650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.152900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.152900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.152900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.152900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152879    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.153104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzkgb\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.153104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.152950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.153104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.153022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.253796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.253796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.253796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.253994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzkgb\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254293    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254569    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.254643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.254553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.256244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.256226    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.256382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.256364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.269190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.269166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.271042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.271022    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzkgb\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb\") pod \"e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") " pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.342640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.342597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:26.382294 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.382258    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:26.645542 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.645513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:35:26.646213 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:26.646186    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e41207_9b1d_4eef_818c_9d666dfe958a.slice/crio-f786a78bbf1741bb20bce7cec8b589d8d0e464fed53b1efcf73f069bb0ed3af9 WatchSource:0}: Error finding container f786a78bbf1741bb20bce7cec8b589d8d0e464fed53b1efcf73f069bb0ed3af9: Status 404 returned error can't find the container with id f786a78bbf1741bb20bce7cec8b589d8d0e464fed53b1efcf73f069bb0ed3af9
Sep 11 13:35:26.648607 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.648569    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:26.648703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.648654    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:26.648703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:26.648697    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:27.026520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.025402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:27.123142 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.123005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:27.435908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.435809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerStarted","Data":"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"}
Sep 11 13:35:27.435908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.435880    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerStarted","Data":"f786a78bbf1741bb20bce7cec8b589d8d0e464fed53b1efcf73f069bb0ed3af9"}
Sep 11 13:35:27.468434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.468362    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podStartSLOduration=2.468340521 podStartE2EDuration="2.468340521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:27.466657936 +0000 UTC m=+851.860345345" watchObservedRunningTime="2026-09-11 13:35:27.468340521 +0000 UTC m=+851.862027934"
Sep 11 13:35:27.512281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.512245    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:27.736863 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.736781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:27.861675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:27.861638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:28.343803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:28.343760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:28.356965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:28.356936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:28.441378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:28.441348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:28.442546 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:28.442486    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:28.788378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:28.788342    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:29.029352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:29.029317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:29.147438 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:29.147353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:30.179899 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:30.179865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:31.292068 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:31.292029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:32.348224 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:32.348190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:32.583999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:32.583961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:32.709242 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:32.709144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:32.909418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:32.909384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:33.013447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.013357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:33.116867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.116835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:33.360312 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.360279    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:33.398316 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.398286    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:35:33.398519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.398362    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:35:33.465559 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.465525    2576 generic.go:356] "Generic (PLEG): container finished" podID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerID="807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea" exitCode=0
Sep 11 13:35:33.465738 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.465589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerDied","Data":"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"}
Sep 11 13:35:33.466044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.466026    2576 scope.go:117] "RemoveContainer" containerID="807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"
Sep 11 13:35:33.467268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.467233    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:33.467380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.467326    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:33.467380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.467375    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:33.843455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.843424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:33.950924 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:33.950814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:34.184070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.183985    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:35:34.186371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.186354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.191274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.191252    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ae1d6b61\""
Sep 11 13:35:34.191408 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.191392    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ae1d6b61-dockercfg-vddpj\""
Sep 11 13:35:34.219902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.219775    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:35:34.331617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.331575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2q4\" (UniqueName: \"kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.331815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.331627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.432697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.432652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2q4\" (UniqueName: \"kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.432697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.432701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.433407 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.433388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.443747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.443685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2q4\" (UniqueName: \"kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4\") pod \"payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") " pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.472058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.472028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerStarted","Data":"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"}
Sep 11 13:35:34.496632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.496602    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:34.600779 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.600750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:34.601082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.601051    2576 patch_prober.go:28] interesting pod/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body=
Sep 11 13:35:34.601155 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.601133    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused"
Sep 11 13:35:34.638558 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.638482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:34.764988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.764961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:34.827536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.818766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:35:34.966334 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:34.966296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:35.216960 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.216875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:35.454417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.454386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:35.478262 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.478166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" event={"ID":"9f178324-d75a-4519-a784-809832d3ec98","Type":"ContainerStarted","Data":"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"}
Sep 11 13:35:35.478262 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.478223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" event={"ID":"9f178324-d75a-4519-a784-809832d3ec98","Type":"ContainerStarted","Data":"3e9e286fd9400fbdf6755784d82df1bd2877eb52d58860a2ef8433384d80bd3a"}
Sep 11 13:35:35.478518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.478473    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:35.600412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.600379    2576 patch_prober.go:28] interesting pod/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body=
Sep 11 13:35:35.600610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.600434    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused"
Sep 11 13:35:35.629564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.629475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:35.821053 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.821012    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:35.937913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.937843    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" podStartSLOduration=1.937816093 podStartE2EDuration="1.937816093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:35.518242517 +0000 UTC m=+859.911929930" watchObservedRunningTime="2026-09-11 13:35:35.937816093 +0000 UTC m=+860.331503521"
Sep 11 13:35:35.940173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.940135    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:35:35.943674 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.943642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:35.946969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.946947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-a649d3-openshift-default-dockercfg-4j9ss\""
Sep 11 13:35:35.960476 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.960452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:35:35.968576 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:35.968526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:36.050046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.049998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx52s\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.050416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.050349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx52s\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151719    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.151885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.151820    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.152386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.152361    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.153886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.153861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.154235 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.154214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.167586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.167562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.176455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.176428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx52s\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s\") pod \"e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") " pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.255007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.254979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:36.331230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.329438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:36.512615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.509975    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:36.575210 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:36.575178    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4f3b99_6e8d_440a_a067_e687bf7c21ab.slice/crio-cf03eb01c6223ab60390866d49a3721c16d5be199e2e32377565236a3657ede2 WatchSource:0}: Error finding container cf03eb01c6223ab60390866d49a3721c16d5be199e2e32377565236a3657ede2: Status 404 returned error can't find the container with id cf03eb01c6223ab60390866d49a3721c16d5be199e2e32377565236a3657ede2
Sep 11 13:35:36.578224 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.578186    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:36.578346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.578312    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:36.578417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.578362    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:36.579843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.579814    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:35:36.600914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.600886    2576 patch_prober.go:28] interesting pod/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body=
Sep 11 13:35:36.601019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.600937    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused"
Sep 11 13:35:36.631239 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:36.631200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:37.489169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.489132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerStarted","Data":"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"}
Sep 11 13:35:37.489367 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.489186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerStarted","Data":"cf03eb01c6223ab60390866d49a3721c16d5be199e2e32377565236a3657ede2"}
Sep 11 13:35:37.514911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.514841    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podStartSLOduration=2.514817115 podStartE2EDuration="2.514817115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:37.51238096 +0000 UTC m=+861.906068371" watchObservedRunningTime="2026-09-11 13:35:37.514817115 +0000 UTC m=+861.908504526"
Sep 11 13:35:37.600599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.600557    2576 patch_prober.go:28] interesting pod/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body=
Sep 11 13:35:37.600749 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.600620    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused"
Sep 11 13:35:37.906692 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:37.906553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:38.256160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.256083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:38.261227 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.261199    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:38.396630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.396576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:38.495333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.495303    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:38.496433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.496413    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:35:38.600519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.600470    2576 patch_prober.go:28] interesting pod/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused" start-of-body=
Sep 11 13:35:38.600918 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:38.600568    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.23:15021/healthz/ready\": dial tcp 10.132.0.23:15021: connect: connection refused"
Sep 11 13:35:39.333231 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.333185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:39.517530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.517477    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:39.628099 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.628023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:39.704070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.704038    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:39.704342 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.704324    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:39.705177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.705156    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:35:39.991582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:39.991430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:40.094021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:40.093983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:40.447615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:40.447587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:40.546676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:40.546648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:40.648581 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:40.648554    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:41.493967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:41.493933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:42.916196 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:42.916161    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:44.174699 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:44.174663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:44.292325 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:44.292290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:44.378304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:44.378270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:44.561550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:44.561463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:45.737207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:45.737166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:46.486975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:46.486951    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:35:46.997420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:46.997386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:47.101649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:47.101616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:47.367999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:47.367961    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:47.846752 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:47.846722    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:47.948105 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:47.948076    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:48.171774 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.170074    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:35:48.173161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.173137    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.176466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.176299    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a649d3-dockercfg-sdp64\""
Sep 11 13:35:48.176466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.176365    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a649d3\""
Sep 11 13:35:48.187397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.187369    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:35:48.271382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.271328    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.271614 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.271528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xclj4\" (UniqueName: \"kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.373027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.372977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xclj4\" (UniqueName: \"kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.373229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.373054    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.373759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.373736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.383649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.383621    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xclj4\" (UniqueName: \"kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4\") pod \"payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.474111 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.473991    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:48.487867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.487829    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:48.771544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:48.771515    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:35:48.775017 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:35:48.774986    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0738c803_8ef9_4bb2_ac3c_fc16dbefca0f.slice/crio-8ac64958a38399918c007ac55af69f13d8acc8148e97a8313371b781a443561e WatchSource:0}: Error finding container 8ac64958a38399918c007ac55af69f13d8acc8148e97a8313371b781a443561e: Status 404 returned error can't find the container with id 8ac64958a38399918c007ac55af69f13d8acc8148e97a8313371b781a443561e
Sep 11 13:35:49.566635 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:49.566595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" event={"ID":"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f","Type":"ContainerStarted","Data":"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"}
Sep 11 13:35:49.567071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:49.566648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" event={"ID":"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f","Type":"ContainerStarted","Data":"8ac64958a38399918c007ac55af69f13d8acc8148e97a8313371b781a443561e"}
Sep 11 13:35:49.567071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:49.566688    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:35:49.591816 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:49.591767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" podStartSLOduration=1.591750787 podStartE2EDuration="1.591750787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:49.591459975 +0000 UTC m=+873.985147387" watchObservedRunningTime="2026-09-11 13:35:49.591750787 +0000 UTC m=+873.985438246"
Sep 11 13:35:49.815747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:49.815717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:50.572753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.572723    2576 generic.go:356] "Generic (PLEG): container finished" podID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerID="1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb" exitCode=0
Sep 11 13:35:50.574179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.572796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerDied","Data":"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"}
Sep 11 13:35:50.574179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.573209    2576 scope.go:117] "RemoveContainer" containerID="1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"
Sep 11 13:35:50.574520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.574419    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:50.574561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.574515    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:50.574594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:50.574562    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:51.211313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.211273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:51.579515 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.579458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerStarted","Data":"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"}
Sep 11 13:35:51.581046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.581016    2576 generic.go:356] "Generic (PLEG): container finished" podID="31f269c6-c533-4037-a224-8e2b2afb903a" containerID="c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e" exitCode=0
Sep 11 13:35:51.581163 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.581089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerDied","Data":"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"}
Sep 11 13:35:51.581434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.581419    2576 scope.go:117] "RemoveContainer" containerID="c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"
Sep 11 13:35:51.582582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.582557    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:51.582651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.582633    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:51.582703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:51.582684    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:35:52.024519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.024452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:52.129228 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.129189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:52.234301 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.234238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:52.343403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.343301    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:52.343972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.343943    2576 patch_prober.go:28] interesting pod/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body=
Sep 11 13:35:52.344172 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.344148    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused"
Sep 11 13:35:52.588204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.588172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerStarted","Data":"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"}
Sep 11 13:35:52.743711 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.743616    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:52.744772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.744744    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:52.744978 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:52.744951    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:53.329702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:53.329640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:53.343414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:53.343377    2576 patch_prober.go:28] interesting pod/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body=
Sep 11 13:35:53.343700 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:53.343669    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused"
Sep 11 13:35:53.743961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:53.743865    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:53.744531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:53.744476    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:54.343220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.343182    2576 patch_prober.go:28] interesting pod/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body=
Sep 11 13:35:54.343531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.343475    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused"
Sep 11 13:35:54.401423 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.401388    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:54.479055 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.479008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:54.743338 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.743235    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:54.743338 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:54.743301    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:55.346520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:55.345872    2576 patch_prober.go:28] interesting pod/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body=
Sep 11 13:35:55.346520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:55.345936    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused"
Sep 11 13:35:55.743533 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:55.743406    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:55.743533 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:55.743478    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:55.824519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:55.824398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:56.343470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:56.343014    2576 patch_prober.go:28] interesting pod/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body=
Sep 11 13:35:56.343470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:56.343081    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused"
Sep 11 13:35:56.743989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:56.743890    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:56.744563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:56.744530    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:57.017149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.017115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:57.357914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.357823    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:57.358873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.358852    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:57.359484 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.359467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:35:57.743318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.743210    2576 patch_prober.go:28] interesting pod/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body=
Sep 11 13:35:57.743318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:57.743269    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused"
Sep 11 13:35:58.000433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:58.000351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:58.747391 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:58.747359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:58.748256 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:58.748235    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:35:58.897833 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:58.897801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:35:59.311797 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:35:59.311744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:00.041011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:00.040972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:00.575588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:00.575561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:36:01.295404 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:01.295372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:02.102862 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:02.102825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:03.082948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:03.082913    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:03.788154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:03.788116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:03.891614 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:03.891584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:03.914324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:03.914288    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:36:03.914594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:03.914560    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" podUID="9f178324-d75a-4519-a784-809832d3ec98" containerName="payload-processing" containerID="cri-o://0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea" gracePeriod=30
Sep 11 13:36:04.005332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.005303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.073336 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.073139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.201713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.201680    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:36:04.201951 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.201926    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy" containerID="cri-o://25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0" gracePeriod=30
Sep 11 13:36:04.203259 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.203177    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:36:04.212739 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.212660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.340221 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.340125    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume\") pod \"9f178324-d75a-4519-a784-809832d3ec98\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") "
Sep 11 13:36:04.340392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.340232    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx2q4\" (UniqueName: \"kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4\") pod \"9f178324-d75a-4519-a784-809832d3ec98\" (UID: \"9f178324-d75a-4519-a784-809832d3ec98\") "
Sep 11 13:36:04.340630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.340597    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f178324-d75a-4519-a784-809832d3ec98" (UID: "9f178324-d75a-4519-a784-809832d3ec98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:04.343362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.343326    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4" (OuterVolumeSpecName: "kube-api-access-lx2q4") pod "9f178324-d75a-4519-a784-809832d3ec98" (UID: "9f178324-d75a-4519-a784-809832d3ec98"). InnerVolumeSpecName "kube-api-access-lx2q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:04.441733 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.441695    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx2q4\" (UniqueName: \"kubernetes.io/projected/9f178324-d75a-4519-a784-809832d3ec98-kube-api-access-lx2q4\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.441733 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.441728    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f178324-d75a-4519-a784-809832d3ec98-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.595070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.594983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.660996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.660966    2576 generic.go:356] "Generic (PLEG): container finished" podID="9f178324-d75a-4519-a784-809832d3ec98" containerID="0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea" exitCode=0
Sep 11 13:36:04.661181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.661025    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"
Sep 11 13:36:04.661181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.661029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" event={"ID":"9f178324-d75a-4519-a784-809832d3ec98","Type":"ContainerDied","Data":"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"}
Sep 11 13:36:04.661181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.661158    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r" event={"ID":"9f178324-d75a-4519-a784-809832d3ec98","Type":"ContainerDied","Data":"3e9e286fd9400fbdf6755784d82df1bd2877eb52d58860a2ef8433384d80bd3a"}
Sep 11 13:36:04.661376 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.661205    2576 scope.go:117] "RemoveContainer" containerID="0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"
Sep 11 13:36:04.671079 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.671060    2576 scope.go:117] "RemoveContainer" containerID="0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"
Sep 11 13:36:04.671349 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:36:04.671329    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea\": container with ID starting with 0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea not found: ID does not exist" containerID="0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"
Sep 11 13:36:04.671442 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.671357    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea"} err="failed to get container status \"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea\": rpc error: code = NotFound desc = could not find container \"0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea\": container with ID starting with 0aa7627a8a6403197bc1c0485a6bfc87c6419a0e9a65710fcec8ae18be5b58ea not found: ID does not exist"
Sep 11 13:36:04.709246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.709207    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:36:04.716605 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.716576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.731797 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.731768    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae1d6b61-6c6cc95bc-7ld7r"]
Sep 11 13:36:04.801326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.801297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:04.959411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:04.959323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:05.360088 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.360053    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:36:05.360649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.360627    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f178324-d75a-4519-a784-809832d3ec98" containerName="payload-processing"
Sep 11 13:36:05.360649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.360649    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f178324-d75a-4519-a784-809832d3ec98" containerName="payload-processing"
Sep 11 13:36:05.360851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.360776    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f178324-d75a-4519-a784-809832d3ec98" containerName="payload-processing"
Sep 11 13:36:05.362435 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:36:05.362412    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4f3b99_6e8d_440a_a067_e687bf7c21ab.slice/crio-conmon-3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 13:36:05.362577 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:36:05.362447    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4f3b99_6e8d_440a_a067_e687bf7c21ab.slice/crio-conmon-3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 13:36:05.363175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.363156    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.367151 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.367132    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-5d979b8c-gw-openshift-default-dockercfg-r69hl\""
Sep 11 13:36:05.386599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.386571    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:36:05.395449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.395423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:05.450877 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.450833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2b62\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.450942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451092 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451221    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.451382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.451302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2b62\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552564 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.552844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.552800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.553058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.553020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.553117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.553063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.553233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.553205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.553437 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.553411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.554949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.554928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.555180 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.555163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.566447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.566384    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.566447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.566428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2b62\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62\") pod \"e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") " pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.668021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.667979    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerID="3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40" exitCode=0
Sep 11 13:36:05.668218 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.668075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerDied","Data":"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"}
Sep 11 13:36:05.668639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.668617    2576 scope.go:117] "RemoveContainer" containerID="3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"
Sep 11 13:36:05.669771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.669741    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:05.669851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.669807    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:05.669851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.669837    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:05.699973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.699937    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:05.726694 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:05.726666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:06.019999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.019960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:36:06.026750 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:36:06.026723    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5268f123_6a11_47fb_9320_f42f6f2cc49a.slice/crio-ee7b283eb05c82477954cb063d4668e17cbd6fcbdd0ea26cb2899816de70a028 WatchSource:0}: Error finding container ee7b283eb05c82477954cb063d4668e17cbd6fcbdd0ea26cb2899816de70a028: Status 404 returned error can't find the container with id ee7b283eb05c82477954cb063d4668e17cbd6fcbdd0ea26cb2899816de70a028
Sep 11 13:36:06.028650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.028625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:06.029026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.028994    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:06.029080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.029066    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:06.029129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.029117    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:06.253542 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.253483    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f178324-d75a-4519-a784-809832d3ec98" path="/var/lib/kubelet/pods/9f178324-d75a-4519-a784-809832d3ec98/volumes"
Sep 11 13:36:06.676072 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.676035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerStarted","Data":"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"}
Sep 11 13:36:06.677522 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.677484    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerStarted","Data":"8600070aeba2102f87ee4a333219dff365408ddedd8ed24a19e989beec15d923"}
Sep 11 13:36:06.677633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.677541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerStarted","Data":"ee7b283eb05c82477954cb063d4668e17cbd6fcbdd0ea26cb2899816de70a028"}
Sep 11 13:36:06.701052 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:06.701025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:07.256110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:07.256073    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:36:07.257307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:07.257280    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:36:07.257428 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:07.257330    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 13:36:07.566769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:07.566711    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podStartSLOduration=2.566695101 podStartE2EDuration="2.566695101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:07.564444865 +0000 UTC m=+891.958132277" watchObservedRunningTime="2026-09-11 13:36:07.566695101 +0000 UTC m=+891.960382512"
Sep 11 13:36:07.705516 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:07.705461    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:08.256255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:08.256176    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:36:08.256255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:08.256230    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 13:36:08.495780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:08.495745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:36:08.687386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:08.687351    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:08.688212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:08.688186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:09.256016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.255984    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:36:09.256119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.256072    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 13:36:09.483216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.483190    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:36:09.590097 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590062    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzkgb\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590112    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590158    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590235    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590289    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590332    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo\") pod \"f0e41207-9b1d-4eef-818c-9d666dfe958a\" (UID: \"f0e41207-9b1d-4eef-818c-9d666dfe958a\") "
Sep 11 13:36:09.590569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:09.590714 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590684    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e41207-9b1d-4eef-818c-9d666dfe958a-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.590770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:09.590770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590728    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:09.590873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:09.590873 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.590841    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:09.592558 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.592535    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:09.592701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.592635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb" (OuterVolumeSpecName: "kube-api-access-xzkgb") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "kube-api-access-xzkgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:09.592799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.592784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:36:09.592838 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.592801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0e41207-9b1d-4eef-818c-9d666dfe958a" (UID: "f0e41207-9b1d-4eef-818c-9d666dfe958a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:09.692149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692112    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzkgb\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-kube-api-access-xzkgb\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692149    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692167    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692180    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692197    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692210    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692223    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e41207-9b1d-4eef-818c-9d666dfe958a-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.692356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.692237    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e41207-9b1d-4eef-818c-9d666dfe958a-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:36:09.695474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.695448    2576 generic.go:356] "Generic (PLEG): container finished" podID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerID="25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0" exitCode=0
Sep 11 13:36:09.695554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.695510    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"
Sep 11 13:36:09.695554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.695529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerDied","Data":"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"}
Sep 11 13:36:09.695616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.695582    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95" event={"ID":"f0e41207-9b1d-4eef-818c-9d666dfe958a","Type":"ContainerDied","Data":"f786a78bbf1741bb20bce7cec8b589d8d0e464fed53b1efcf73f069bb0ed3af9"}
Sep 11 13:36:09.695616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.695605    2576 scope.go:117] "RemoveContainer" containerID="25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"
Sep 11 13:36:09.705039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.704955    2576 scope.go:117] "RemoveContainer" containerID="1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"
Sep 11 13:36:09.730310 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.730050    2576 scope.go:117] "RemoveContainer" containerID="25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"
Sep 11 13:36:09.730676 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:36:09.730646    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0\": container with ID starting with 25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0 not found: ID does not exist" containerID="25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"
Sep 11 13:36:09.730781 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.730690    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0"} err="failed to get container status \"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0\": rpc error: code = NotFound desc = could not find container \"25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0\": container with ID starting with 25d864b73ef8ab0c8654a1cad45dcfc2ce837ae87cdadbb9bd862e6af15066f0 not found: ID does not exist"
Sep 11 13:36:09.730781 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.730719    2576 scope.go:117] "RemoveContainer" containerID="1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"
Sep 11 13:36:09.731078 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:36:09.731058    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb\": container with ID starting with 1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb not found: ID does not exist" containerID="1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"
Sep 11 13:36:09.731169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:09.731087    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb"} err="failed to get container status \"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb\": rpc error: code = NotFound desc = could not find container \"1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb\": container with ID starting with 1e8c81ab292311a197e0b35e55167d2a73cb421f12cd406215c91b1c95542edb not found: ID does not exist"
Sep 11 13:36:10.255595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:10.255560    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:36:10.255771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:10.255621    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 13:36:11.256301 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:11.256263    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:36:11.256691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:11.256322    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Sep 11 13:36:12.260340 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:12.260309    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:36:12.261365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:12.261345    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:36:25.461870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:25.461830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:25.463349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:25.463325    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:36:25.467591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:25.467562    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ae1d6b61-openshift-default-5bbfbd9f96-hbn95"]
Sep 11 13:36:26.248017 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:26.247972    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" path="/var/lib/kubelet/pods/f0e41207-9b1d-4eef-818c-9d666dfe958a/volumes"
Sep 11 13:36:33.325852 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:33.325802    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:33.395745 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:33.395700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:33.784896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:33.784852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:34.490050 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:34.490020    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:34.589872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:34.589838    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:34.811347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:34.811292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:35.910753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:35.910720    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:36.866224 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:36.866175    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.038591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.038539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.129667 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.129633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.673457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.673404    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.775544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.775508    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.800753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.800720    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:36:38.801177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801156    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.801328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801273    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.801328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801300    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.801328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801309    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.801528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801405    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.801528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.801422    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e41207-9b1d-4eef-818c-9d666dfe958a" containerName="istio-proxy"
Sep 11 13:36:38.804541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.804518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.807084 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.807055    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-5d979b8c\""
Sep 11 13:36:38.807084 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.807083    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-5d979b8c-dockercfg-zkr4x\""
Sep 11 13:36:38.815722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.815695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:36:38.851373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.851336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g9g9\" (UniqueName: \"kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.851602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.851403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.907648 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.907606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:38.952977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.952875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g9g9\" (UniqueName: \"kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.952977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.952958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.953679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.953660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:38.960688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:38.960654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g9g9\" (UniqueName: \"kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9\") pod \"payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") " pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:39.116234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.116193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:39.208538 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.208455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:39.391447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.391421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:36:39.394143 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:36:39.394115    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbaf7c7_cee3_427b_98f4_983a1780051c.slice/crio-ab00f1bd7f3879042f2127e8e7645912d28df495589627ee444ebf3ba8503d3f WatchSource:0}: Error finding container ab00f1bd7f3879042f2127e8e7645912d28df495589627ee444ebf3ba8503d3f: Status 404 returned error can't find the container with id ab00f1bd7f3879042f2127e8e7645912d28df495589627ee444ebf3ba8503d3f
Sep 11 13:36:39.834040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.834001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" event={"ID":"3fbaf7c7-cee3-427b-98f4-983a1780051c","Type":"ContainerStarted","Data":"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"}
Sep 11 13:36:39.834243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.834066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" event={"ID":"3fbaf7c7-cee3-427b-98f4-983a1780051c","Type":"ContainerStarted","Data":"ab00f1bd7f3879042f2127e8e7645912d28df495589627ee444ebf3ba8503d3f"}
Sep 11 13:36:39.834243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.834139    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:39.853158 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:39.853105    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" podStartSLOduration=1.853088197 podStartE2EDuration="1.853088197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:36:39.850934064 +0000 UTC m=+924.244621476" watchObservedRunningTime="2026-09-11 13:36:39.853088197 +0000 UTC m=+924.246775608"
Sep 11 13:36:40.206917 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:40.206836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:41.208542 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:41.208512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:43.294771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:43.294739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:44.633653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:44.633620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:36:50.842009 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:50.841954    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:36:57.922790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.922755    2576 generic.go:356] "Generic (PLEG): container finished" podID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerID="8600070aeba2102f87ee4a333219dff365408ddedd8ed24a19e989beec15d923" exitCode=0
Sep 11 13:36:57.923226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.922831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerDied","Data":"8600070aeba2102f87ee4a333219dff365408ddedd8ed24a19e989beec15d923"}
Sep 11 13:36:57.923279 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.923226    2576 scope.go:117] "RemoveContainer" containerID="8600070aeba2102f87ee4a333219dff365408ddedd8ed24a19e989beec15d923"
Sep 11 13:36:57.924288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.924261    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:57.924341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.924327    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:57.924384 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:57.924357    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:36:58.932100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:58.932063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerStarted","Data":"c2eeafa2268a12b6257b826d401597fa2d69b55954854777f0d55df47bedebb7"}
Sep 11 13:36:59.700489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:59.700445    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:36:59.700705 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:59.700681    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 11 13:36:59.700751 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:36:59.700734    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 11 13:37:00.700668 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:00.700634    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 11 13:37:00.701058 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:00.700696    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 11 13:37:00.705336 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:00.705305    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:01.642885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:01.642847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:01.700791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:01.700750    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 11 13:37:01.701730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:01.701682    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 11 13:37:02.607796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:02.607729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:02.701171 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:02.701134    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 11 13:37:02.701563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:02.701189    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 11 13:37:03.394104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:03.394066    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:03.700914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:03.700831    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Sep 11 13:37:03.700914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:03.700885    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Sep 11 13:37:04.424928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:04.424882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:04.704926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:04.704844    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:37:04.705171 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:04.705145    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:37:04.705840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:04.705821    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:37:05.489337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:05.489290    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:06.477609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:06.477577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:07.353096 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:07.353053    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:08.309722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:08.309690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:09.295082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:09.295052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:10.099692 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:10.099660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:10.893974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:10.893947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:14.875608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:14.875574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:15.432999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:15.432965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.314443 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.314409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.419677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.419632    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:37:16.419893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.419831    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" podUID="3fbaf7c7-cee3-427b-98f4-983a1780051c" containerName="payload-processing" containerID="cri-o://991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61" gracePeriod=30
Sep 11 13:37:16.473065 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.473035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.572885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.572799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.667988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.667961    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:37:16.698281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.698254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.722898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.722871    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g9g9\" (UniqueName: \"kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9\") pod \"3fbaf7c7-cee3-427b-98f4-983a1780051c\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") "
Sep 11 13:37:16.723087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.722923    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume\") pod \"3fbaf7c7-cee3-427b-98f4-983a1780051c\" (UID: \"3fbaf7c7-cee3-427b-98f4-983a1780051c\") "
Sep 11 13:37:16.723331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.723306    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3fbaf7c7-cee3-427b-98f4-983a1780051c" (UID: "3fbaf7c7-cee3-427b-98f4-983a1780051c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:16.725005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.724982    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9" (OuterVolumeSpecName: "kube-api-access-4g9g9") pod "3fbaf7c7-cee3-427b-98f4-983a1780051c" (UID: "3fbaf7c7-cee3-427b-98f4-983a1780051c"). InnerVolumeSpecName "kube-api-access-4g9g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:16.766675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.766642    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:16.824262 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.824135    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g9g9\" (UniqueName: \"kubernetes.io/projected/3fbaf7c7-cee3-427b-98f4-983a1780051c-kube-api-access-4g9g9\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:16.824262 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.824175    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fbaf7c7-cee3-427b-98f4-983a1780051c-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:16.905274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.905239    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:37:16.905477 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.905456    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" containerID="cri-o://c2eeafa2268a12b6257b826d401597fa2d69b55954854777f0d55df47bedebb7" gracePeriod=30
Sep 11 13:37:16.911986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:16.911956    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:17.021461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.021426    2576 generic.go:356] "Generic (PLEG): container finished" podID="3fbaf7c7-cee3-427b-98f4-983a1780051c" containerID="991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61" exitCode=0
Sep 11 13:37:17.021680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.021525    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" event={"ID":"3fbaf7c7-cee3-427b-98f4-983a1780051c","Type":"ContainerDied","Data":"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"}
Sep 11 13:37:17.021680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.021577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2" event={"ID":"3fbaf7c7-cee3-427b-98f4-983a1780051c","Type":"ContainerDied","Data":"ab00f1bd7f3879042f2127e8e7645912d28df495589627ee444ebf3ba8503d3f"}
Sep 11 13:37:17.021680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.021608    2576 scope.go:117] "RemoveContainer" containerID="991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"
Sep 11 13:37:17.021850 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.021778    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"
Sep 11 13:37:17.031480 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.031460    2576 scope.go:117] "RemoveContainer" containerID="991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"
Sep 11 13:37:17.031796 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:37:17.031776    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61\": container with ID starting with 991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61 not found: ID does not exist" containerID="991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"
Sep 11 13:37:17.031864 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.031808    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61"} err="failed to get container status \"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61\": rpc error: code = NotFound desc = could not find container \"991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61\": container with ID starting with 991b516c1fc13fcee555c945dea58302c91f24a55610da8dbbfb786784afab61 not found: ID does not exist"
Sep 11 13:37:17.049527 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.049475    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:37:17.067268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.067237    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-5d979b8c-68b76846c4-jz8p2"]
Sep 11 13:37:17.409630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.409595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:17.531682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.531650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:17.870281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.870238    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:37:17.870682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.870663    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbaf7c7-cee3-427b-98f4-983a1780051c" containerName="payload-processing"
Sep 11 13:37:17.870787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.870683    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbaf7c7-cee3-427b-98f4-983a1780051c" containerName="payload-processing"
Sep 11 13:37:17.870855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.870788    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fbaf7c7-cee3-427b-98f4-983a1780051c" containerName="payload-processing"
Sep 11 13:37:17.873705 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.873683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.882065 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.882044    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eeb77b9d-openshift-default-dockercfg-n7g4x\""
Sep 11 13:37:17.892666 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.892640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:17.935417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26zx\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.935988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.935726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:17.974336 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:17.974298    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:37:18.036652 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.036827 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.036827 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.036911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.036911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.036974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036936    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037015 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.036983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037041    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037103    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26zx\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037283 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037283 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.037806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.037781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.039030 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.039010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.039116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.039099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.050067 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.050040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.052104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.052083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26zx\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx\") pod \"e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") " pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.183439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.183346    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:18.249925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.249894    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fbaf7c7-cee3-427b-98f4-983a1780051c" path="/var/lib/kubelet/pods/3fbaf7c7-cee3-427b-98f4-983a1780051c/volumes"
Sep 11 13:37:18.263574 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.263491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:18.462569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.462546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:37:18.464222 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:37:18.464198    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78f726be_6435_40d2_bbbb_1a93c07c4658.slice/crio-11bbbb1ed8a8562b1332716686533f3298543e998dfe4f4ec7685dce9a49dc63 WatchSource:0}: Error finding container 11bbbb1ed8a8562b1332716686533f3298543e998dfe4f4ec7685dce9a49dc63: Status 404 returned error can't find the container with id 11bbbb1ed8a8562b1332716686533f3298543e998dfe4f4ec7685dce9a49dc63
Sep 11 13:37:18.466516 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.466465    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:18.466613 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.466556    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:18.466613 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.466583    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:18.817444 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:18.817411    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:19.008891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.008857    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"]
Sep 11 13:37:19.011440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.011418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:19.014095 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.014069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:37:19.014207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.014102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:37:19.022410 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.022389    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"]
Sep 11 13:37:19.033544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.033486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerStarted","Data":"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"}
Sep 11 13:37:19.033671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.033568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerStarted","Data":"11bbbb1ed8a8562b1332716686533f3298543e998dfe4f4ec7685dce9a49dc63"}
Sep 11 13:37:19.046367 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.046333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl\" (UID: \"6b189c58-0d35-4c57-a8dd-d93c229478eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:19.066648 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.066598    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podStartSLOduration=2.066584947 podStartE2EDuration="2.066584947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:19.063867627 +0000 UTC m=+963.457555038" watchObservedRunningTime="2026-09-11 13:37:19.066584947 +0000 UTC m=+963.460272355"
Sep 11 13:37:19.147931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.147845    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl\" (UID: \"6b189c58-0d35-4c57-a8dd-d93c229478eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:19.148461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.148443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl\" (UID: \"6b189c58-0d35-4c57-a8dd-d93c229478eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:19.184463 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.184421    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:19.196330 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.196301    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:19.322027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.321990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:19.591231 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.591207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"]
Sep 11 13:37:19.592956 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:37:19.592921    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b189c58_0d35_4c57_a8dd_d93c229478eb.slice/crio-968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a WatchSource:0}: Error finding container 968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a: Status 404 returned error can't find the container with id 968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a
Sep 11 13:37:19.707773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.707735    2576 patch_prober.go:28] interesting pod/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:37:19.707937 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:19.707795    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:37:20.038796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.038761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl" event={"ID":"6b189c58-0d35-4c57-a8dd-d93c229478eb","Type":"ContainerStarted","Data":"968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a"}
Sep 11 13:37:20.038974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.038963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:20.040008 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.039985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:20.108331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.108284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:20.448274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.448191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:20.538335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:20.538302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:21.398893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:21.398861    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:22.053465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.053436    2576 generic.go:356] "Generic (PLEG): container finished" podID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerID="c2eeafa2268a12b6257b826d401597fa2d69b55954854777f0d55df47bedebb7" exitCode=0
Sep 11 13:37:22.053646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.053537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerDied","Data":"c2eeafa2268a12b6257b826d401597fa2d69b55954854777f0d55df47bedebb7"}
Sep 11 13:37:22.053646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.053615    2576 scope.go:117] "RemoveContainer" containerID="8600070aeba2102f87ee4a333219dff365408ddedd8ed24a19e989beec15d923"
Sep 11 13:37:22.144118 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.144092    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:37:22.177715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177689    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177732    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2b62\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177862    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177890    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.177944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177930    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.178240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.177986    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data\") pod \"5268f123-6a11-47fb-9320-f42f6f2cc49a\" (UID: \"5268f123-6a11-47fb-9320-f42f6f2cc49a\") "
Sep 11 13:37:22.178240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178160    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:22.178346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178237    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:22.178346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178273    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:22.178445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178354    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5268f123-6a11-47fb-9320-f42f6f2cc49a-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.178445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178402    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.178445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data" (OuterVolumeSpecName: "istio-data") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:22.178445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.178430    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:22.180454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.180431    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token" (OuterVolumeSpecName: "istio-token") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:22.180454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.180428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:37:22.180676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.180656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62" (OuterVolumeSpecName: "kube-api-access-l2b62") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "kube-api-access-l2b62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:22.180745 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.180730    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5268f123-6a11-47fb-9320-f42f6f2cc49a" (UID: "5268f123-6a11-47fb-9320-f42f6f2cc49a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:22.279236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279187    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279235    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279250    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279263    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2b62\" (UniqueName: \"kubernetes.io/projected/5268f123-6a11-47fb-9320-f42f6f2cc49a-kube-api-access-l2b62\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279277    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5268f123-6a11-47fb-9320-f42f6f2cc49a-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279290    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.279530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.279303    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5268f123-6a11-47fb-9320-f42f6f2cc49a-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:22.637387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:22.637349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:23.060033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.059995    2576 generic.go:356] "Generic (PLEG): container finished" podID="6b189c58-0d35-4c57-a8dd-d93c229478eb" containerID="cb56410166171726feb37006e3a03a0fd17a433abb68acf657a4d3ddf9ff7986" exitCode=0
Sep 11 13:37:23.060215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.060074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl" event={"ID":"6b189c58-0d35-4c57-a8dd-d93c229478eb","Type":"ContainerDied","Data":"cb56410166171726feb37006e3a03a0fd17a433abb68acf657a4d3ddf9ff7986"}
Sep 11 13:37:23.061347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.061329    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"
Sep 11 13:37:23.061347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.061341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl" event={"ID":"5268f123-6a11-47fb-9320-f42f6f2cc49a","Type":"ContainerDied","Data":"ee7b283eb05c82477954cb063d4668e17cbd6fcbdd0ea26cb2899816de70a028"}
Sep 11 13:37:23.061474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.061391    2576 scope.go:117] "RemoveContainer" containerID="c2eeafa2268a12b6257b826d401597fa2d69b55954854777f0d55df47bedebb7"
Sep 11 13:37:23.101394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.101360    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:37:23.104027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.104002    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-5d979b8c-gw-openshift-default-76b967d49b-6z5cl"]
Sep 11 13:37:23.724281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:23.724252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:24.203573 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.203546    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:24.250090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.250054    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" path="/var/lib/kubelet/pods/5268f123-6a11-47fb-9320-f42f6f2cc49a/volumes"
Sep 11 13:37:24.297932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.297897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca\") pod \"6b189c58-0d35-4c57-a8dd-d93c229478eb\" (UID: \"6b189c58-0d35-4c57-a8dd-d93c229478eb\") "
Sep 11 13:37:24.298270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.298245    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6b189c58-0d35-4c57-a8dd-d93c229478eb" (UID: "6b189c58-0d35-4c57-a8dd-d93c229478eb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:24.398688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.398604    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b189c58-0d35-4c57-a8dd-d93c229478eb-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:37:24.860166 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:24.860135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:25.075305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:25.075266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl" event={"ID":"6b189c58-0d35-4c57-a8dd-d93c229478eb","Type":"ContainerDied","Data":"968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a"}
Sep 11 13:37:25.075305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:25.075305    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"
Sep 11 13:37:25.075551 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:25.075320    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="968fa9e1e172f2902e4eb32c5e409bf11fa971054b3d8e60d9aecf9d989d3a8a"
Sep 11 13:37:26.134326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:26.134294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:26.225952 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:26.225920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:26.579716 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:26.579671    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:26.903388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:26.903267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:26.998992 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:26.998947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:27.074284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074243    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:37:27.074637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074623    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b189c58-0d35-4c57-a8dd-d93c229478eb" containerName="revoke-keys"
Sep 11 13:37:27.074690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074640    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b189c58-0d35-4c57-a8dd-d93c229478eb" containerName="revoke-keys"
Sep 11 13:37:27.074690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074656    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.074690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074662    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.074690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074682    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.074690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074690    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.074847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074748    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.074847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.074758    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b189c58-0d35-4c57-a8dd-d93c229478eb" containerName="revoke-keys"
Sep 11 13:37:27.076808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.076792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.079087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.079066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-eeb77b9d-dockercfg-hhvlh\""
Sep 11 13:37:27.079450 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.079435    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-eeb77b9d\""
Sep 11 13:37:27.096601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.096567    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:37:27.122932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.122896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.123146 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.122965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hzbm\" (UniqueName: \"kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.124909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.124878    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:37:27.125361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.125343    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5268f123-6a11-47fb-9320-f42f6f2cc49a" containerName="istio-proxy"
Sep 11 13:37:27.127274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.127255    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.140964 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.140926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:37:27.224018 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.223908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.224018 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.223975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.224018 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.224008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hzbm\" (UniqueName: \"kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.224297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.224028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpvq9\" (UniqueName: \"kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.224657 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.224637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.240806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.240772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hzbm\" (UniqueName: \"kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm\") pod \"payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") " pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.324923 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.324884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.324923 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.324931    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpvq9\" (UniqueName: \"kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.325655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.325635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.334094 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.334069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpvq9\" (UniqueName: \"kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9\") pod \"payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.388002 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.387964    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:27.437346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.437303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:27.584901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.584874    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:27.686063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.686037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:37:27.688018 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:37:27.687989    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f655621_735a_433e_a740_3376e4331769.slice/crio-7bf5e3ed67740bcccec9f916a9ba9fb83dac308dfd8bfd5d69da24b3c22d2a15 WatchSource:0}: Error finding container 7bf5e3ed67740bcccec9f916a9ba9fb83dac308dfd8bfd5d69da24b3c22d2a15: Status 404 returned error can't find the container with id 7bf5e3ed67740bcccec9f916a9ba9fb83dac308dfd8bfd5d69da24b3c22d2a15
Sep 11 13:37:27.723801 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:27.723764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:37:27.727199 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:37:27.727170    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5727b806_6012_40b0_a2d2_ca97cd5c2fce.slice/crio-837a7be10edbe76f63e6f904ee8cd205665d92b24fddbfaf99caae19e8d9f99b WatchSource:0}: Error finding container 837a7be10edbe76f63e6f904ee8cd205665d92b24fddbfaf99caae19e8d9f99b: Status 404 returned error can't find the container with id 837a7be10edbe76f63e6f904ee8cd205665d92b24fddbfaf99caae19e8d9f99b
Sep 11 13:37:28.092698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.092654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" event={"ID":"5f655621-735a-433e-a740-3376e4331769","Type":"ContainerStarted","Data":"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"}
Sep 11 13:37:28.092903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.092726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" event={"ID":"5f655621-735a-433e-a740-3376e4331769","Type":"ContainerStarted","Data":"7bf5e3ed67740bcccec9f916a9ba9fb83dac308dfd8bfd5d69da24b3c22d2a15"}
Sep 11 13:37:28.093277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.093249    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:28.095450 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.095418    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" event={"ID":"5727b806-6012-40b0-a2d2-ca97cd5c2fce","Type":"ContainerStarted","Data":"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"}
Sep 11 13:37:28.095600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.095471    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" event={"ID":"5727b806-6012-40b0-a2d2-ca97cd5c2fce","Type":"ContainerStarted","Data":"837a7be10edbe76f63e6f904ee8cd205665d92b24fddbfaf99caae19e8d9f99b"}
Sep 11 13:37:28.095600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.095575    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:28.148626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.148568    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" podStartSLOduration=1.148552026 podStartE2EDuration="1.148552026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:28.119074242 +0000 UTC m=+972.512761655" watchObservedRunningTime="2026-09-11 13:37:28.148552026 +0000 UTC m=+972.542239437"
Sep 11 13:37:28.149007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.148836    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" podStartSLOduration=1.148830499 podStartE2EDuration="1.148830499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:37:28.146596695 +0000 UTC m=+972.540284106" watchObservedRunningTime="2026-09-11 13:37:28.148830499 +0000 UTC m=+972.542517927"
Sep 11 13:37:28.584583 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:28.584553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.312357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.312329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.409981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.409952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.511512 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.511473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.616680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.616589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.630709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.630676    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"]
Sep 11 13:37:29.636308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.636278    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-f9f26e-s6bgl"]
Sep 11 13:37:29.685406 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.685370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:29.966706 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:29.966621    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:30.136949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:30.136919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:30.248218 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:30.248131    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b189c58-0d35-4c57-a8dd-d93c229478eb" path="/var/lib/kubelet/pods/6b189c58-0d35-4c57-a8dd-d93c229478eb/volumes"
Sep 11 13:37:31.126213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:31.126174    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:32.178722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:32.178688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:33.092427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:33.092398    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:33.942071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:33.942040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:34.904659 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:34.904623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:35.115211 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:35.115177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:39.102443 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:39.102364    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:37:39.102819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:39.102548    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:37:47.207145 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.207113    2576 generic.go:356] "Generic (PLEG): container finished" podID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerID="cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67" exitCode=0
Sep 11 13:37:47.207569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.207167    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerDied","Data":"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"}
Sep 11 13:37:47.207569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.207544    2576 scope.go:117] "RemoveContainer" containerID="cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"
Sep 11 13:37:47.208644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.208614    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:47.208708 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.208685    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:47.208746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.208715    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:37:47.501229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:47.501145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:48.214416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:48.214385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerStarted","Data":"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"}
Sep 11 13:37:48.557135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:48.557089    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.139840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:49.139811    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:49.184413 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:49.184377    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:49.185374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:49.185349    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 11 13:37:49.185514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:49.185407    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 11 13:37:50.040006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:50.039969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:50.184773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:50.184735    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 11 13:37:50.184959 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:50.184794    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 11 13:37:50.367507 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:50.367409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:51.184022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:51.183979    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 11 13:37:51.184409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:51.184037    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 11 13:37:51.455131 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:51.455047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:52.184794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:52.184748    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 11 13:37:52.185286 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:52.184814    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 11 13:37:52.437159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:52.437062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:53.183821 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:53.183782    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Sep 11 13:37:53.184003 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:53.183838    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Sep 11 13:37:53.315724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:53.315691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:54.188059 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:54.188028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:54.189230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:54.189208    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:37:54.487891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:54.487805    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:55.430488 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:55.430454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:56.287554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:56.287522    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:37:57.218830 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:37:57.218796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:02.191921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:02.191880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:02.946119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:02.946084    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:38:02.946363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:02.946323    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" podUID="5f655621-735a-433e-a740-3376e4331769" containerName="payload-processing" containerID="cri-o://0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae" gracePeriod=30
Sep 11 13:38:02.956141 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:02.956113    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:38:02.956332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:02.956311    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" podUID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" containerName="payload-pre-processing" containerID="cri-o://0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe" gracePeriod=30
Sep 11 13:38:03.225040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.225018    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:38:03.228125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.228108    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:38:03.294391 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.294351    2576 generic.go:356] "Generic (PLEG): container finished" podID="5f655621-735a-433e-a740-3376e4331769" containerID="0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae" exitCode=0
Sep 11 13:38:03.294612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.294406    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"
Sep 11 13:38:03.295602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.295475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" event={"ID":"5f655621-735a-433e-a740-3376e4331769","Type":"ContainerDied","Data":"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"}
Sep 11 13:38:03.295839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.295805    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz" event={"ID":"5f655621-735a-433e-a740-3376e4331769","Type":"ContainerDied","Data":"7bf5e3ed67740bcccec9f916a9ba9fb83dac308dfd8bfd5d69da24b3c22d2a15"}
Sep 11 13:38:03.296106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.295860    2576 scope.go:117] "RemoveContainer" containerID="0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"
Sep 11 13:38:03.297139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.296722    2576 generic.go:356] "Generic (PLEG): container finished" podID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" containerID="0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe" exitCode=0
Sep 11 13:38:03.297139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.296866    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" event={"ID":"5727b806-6012-40b0-a2d2-ca97cd5c2fce","Type":"ContainerDied","Data":"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"}
Sep 11 13:38:03.297139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.296919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb" event={"ID":"5727b806-6012-40b0-a2d2-ca97cd5c2fce","Type":"ContainerDied","Data":"837a7be10edbe76f63e6f904ee8cd205665d92b24fddbfaf99caae19e8d9f99b"}
Sep 11 13:38:03.297139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.296990    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"
Sep 11 13:38:03.314892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.314820    2576 scope.go:117] "RemoveContainer" containerID="0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"
Sep 11 13:38:03.315278 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:03.315251    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae\": container with ID starting with 0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae not found: ID does not exist" containerID="0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"
Sep 11 13:38:03.315351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.315290    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae"} err="failed to get container status \"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae\": rpc error: code = NotFound desc = could not find container \"0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae\": container with ID starting with 0c709bad6a492f89c0409bcdfe623918d9b95fe92dac96397f2f899631c3b0ae not found: ID does not exist"
Sep 11 13:38:03.315351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.315314    2576 scope.go:117] "RemoveContainer" containerID="0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"
Sep 11 13:38:03.333967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.333279    2576 scope.go:117] "RemoveContainer" containerID="0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"
Sep 11 13:38:03.333967 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:03.333771    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe\": container with ID starting with 0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe not found: ID does not exist" containerID="0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"
Sep 11 13:38:03.333967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.333805    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe"} err="failed to get container status \"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe\": rpc error: code = NotFound desc = could not find container \"0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe\": container with ID starting with 0fac454407644e15d00085e1193797b5430e3dcf779bdd0e34a338cf4480dcfe not found: ID does not exist"
Sep 11 13:38:03.361491 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361450    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume\") pod \"5f655621-735a-433e-a740-3376e4331769\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") "
Sep 11 13:38:03.361707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361554    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hzbm\" (UniqueName: \"kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm\") pod \"5f655621-735a-433e-a740-3376e4331769\" (UID: \"5f655621-735a-433e-a740-3376e4331769\") "
Sep 11 13:38:03.361707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361597    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume\") pod \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") "
Sep 11 13:38:03.361707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361612    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpvq9\" (UniqueName: \"kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9\") pod \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\" (UID: \"5727b806-6012-40b0-a2d2-ca97cd5c2fce\") "
Sep 11 13:38:03.361927 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361863    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f655621-735a-433e-a740-3376e4331769" (UID: "5f655621-735a-433e-a740-3376e4331769"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:03.361927 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.361899    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5727b806-6012-40b0-a2d2-ca97cd5c2fce" (UID: "5727b806-6012-40b0-a2d2-ca97cd5c2fce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:03.363788 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.363758    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm" (OuterVolumeSpecName: "kube-api-access-9hzbm") pod "5f655621-735a-433e-a740-3376e4331769" (UID: "5f655621-735a-433e-a740-3376e4331769"). InnerVolumeSpecName "kube-api-access-9hzbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:03.363966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.363803    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9" (OuterVolumeSpecName: "kube-api-access-kpvq9") pod "5727b806-6012-40b0-a2d2-ca97cd5c2fce" (UID: "5727b806-6012-40b0-a2d2-ca97cd5c2fce"). InnerVolumeSpecName "kube-api-access-kpvq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:03.463163 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.463053    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hzbm\" (UniqueName: \"kubernetes.io/projected/5f655621-735a-433e-a740-3376e4331769-kube-api-access-9hzbm\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:03.463163 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.463098    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5727b806-6012-40b0-a2d2-ca97cd5c2fce-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:03.463163 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.463114    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpvq9\" (UniqueName: \"kubernetes.io/projected/5727b806-6012-40b0-a2d2-ca97cd5c2fce-kube-api-access-kpvq9\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:03.463163 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.463128    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f655621-735a-433e-a740-3376e4331769-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:03.616469 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.616436    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:38:03.618147 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.618125    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eeb77b9d-54ccd85bd5-nvxqz"]
Sep 11 13:38:03.630567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.630541    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:38:03.633822 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.633799    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eeb77b9d-78645c6b58-9vvxb"]
Sep 11 13:38:03.677258 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:03.677224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:04.250878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.250842    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" path="/var/lib/kubelet/pods/5727b806-6012-40b0-a2d2-ca97cd5c2fce/volumes"
Sep 11 13:38:04.251342 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.251330    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f655621-735a-433e-a740-3376e4331769" path="/var/lib/kubelet/pods/5f655621-735a-433e-a740-3376e4331769/volumes"
Sep 11 13:38:04.270238 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.270213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:04.278281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.278255    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:38:04.278444 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.278427    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" containerID="cri-o://97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa" gracePeriod=30
Sep 11 13:38:04.682867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.682835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:04.793615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:04.793585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:05.006548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:05.006458    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:05.187875 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:05.187847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:05.665435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:05.665400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:06.970546 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:06.970482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:07.592562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:07.592523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:07.688837 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:07.688808    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:07.748542 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:07.748483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:07.818299 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:07.818267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:08.751057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:08.751025    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:09.190806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.190771    2576 patch_prober.go:28] interesting pod/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:38:09.190974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.190830    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:38:09.528145 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.528123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:38:09.610998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.610967    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611015    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m26zx\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611116    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611170    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611196    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611218    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611268    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs\") pod \"78f726be-6435-40d2-bbbb-1a93c07c4658\" (UID: \"78f726be-6435-40d2-bbbb-1a93c07c4658\") "
Sep 11 13:38:09.611400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611283    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:09.611682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:09.611682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611586    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.611682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:09.611838 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611769    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:09.611838 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.611784    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data" (OuterVolumeSpecName: "istio-data") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:09.613414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.613385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:09.613528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.613413    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:38:09.613727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.613708    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token" (OuterVolumeSpecName: "istio-token") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:09.613792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.613721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx" (OuterVolumeSpecName: "kube-api-access-m26zx") pod "78f726be-6435-40d2-bbbb-1a93c07c4658" (UID: "78f726be-6435-40d2-bbbb-1a93c07c4658"). InnerVolumeSpecName "kube-api-access-m26zx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:09.712791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712714    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.712791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712745    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m26zx\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-kube-api-access-m26zx\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.712791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712759    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.712791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712773    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78f726be-6435-40d2-bbbb-1a93c07c4658-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.712791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712787    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.713051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712799    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.713051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712811    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.713051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.712823    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78f726be-6435-40d2-bbbb-1a93c07c4658-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:09.734183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:09.734146    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:10.332246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.332209    2576 generic.go:356] "Generic (PLEG): container finished" podID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerID="97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa" exitCode=0
Sep 11 13:38:10.332677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.332266    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"
Sep 11 13:38:10.332677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.332296    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerDied","Data":"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"}
Sep 11 13:38:10.332677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.332354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v" event={"ID":"78f726be-6435-40d2-bbbb-1a93c07c4658","Type":"ContainerDied","Data":"11bbbb1ed8a8562b1332716686533f3298543e998dfe4f4ec7685dce9a49dc63"}
Sep 11 13:38:10.332677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.332380    2576 scope.go:117] "RemoveContainer" containerID="97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"
Sep 11 13:38:10.340698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.340678    2576 scope.go:117] "RemoveContainer" containerID="cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"
Sep 11 13:38:10.348414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.348396    2576 scope.go:117] "RemoveContainer" containerID="97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"
Sep 11 13:38:10.348696 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:10.348677    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa\": container with ID starting with 97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa not found: ID does not exist" containerID="97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"
Sep 11 13:38:10.348757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.348705    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa"} err="failed to get container status \"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa\": rpc error: code = NotFound desc = could not find container \"97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa\": container with ID starting with 97658f7da0fa303022737dc54a6b0fc4ba19ec824472ef9d5cdcb7408b02b0fa not found: ID does not exist"
Sep 11 13:38:10.348757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.348722    2576 scope.go:117] "RemoveContainer" containerID="cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"
Sep 11 13:38:10.348975 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:10.348957    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67\": container with ID starting with cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67 not found: ID does not exist" containerID="cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"
Sep 11 13:38:10.349028 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.348983    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67"} err="failed to get container status \"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67\": rpc error: code = NotFound desc = could not find container \"cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67\": container with ID starting with cc59c96fb56a073d1d57fcd2aefaaf4653762cc80ed1340563c76f4b52ff6d67 not found: ID does not exist"
Sep 11 13:38:10.359173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.359147    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:38:10.365178 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.365154    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eeb77b9d-openshift-default-5c995ffdf4-z7f2v"]
Sep 11 13:38:10.689051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:10.689023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:11.516309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:11.516276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:12.248478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:12.248435    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" path="/var/lib/kubelet/pods/78f726be-6435-40d2-bbbb-1a93c07c4658/volumes"
Sep 11 13:38:12.405864 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:12.405817    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:12.494341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:12.494297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:12.593015 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:12.592983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:12.960315 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:12.960221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:13.058759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.058728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:13.100993 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.100950    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:13.101317 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101302    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:13.101369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:13.101369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101329    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f655621-735a-433e-a740-3376e4331769" containerName="payload-processing"
Sep 11 13:38:13.101369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101335    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f655621-735a-433e-a740-3376e4331769" containerName="payload-processing"
Sep 11 13:38:13.101369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101349    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" containerName="payload-pre-processing"
Sep 11 13:38:13.101369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101355    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" containerName="payload-pre-processing"
Sep 11 13:38:13.101570 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101417    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:13.101570 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101423    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:13.101570 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101430    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f655621-735a-433e-a740-3376e4331769" containerName="payload-processing"
Sep 11 13:38:13.101570 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.101438    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5727b806-6012-40b0-a2d2-ca97cd5c2fce" containerName="payload-pre-processing"
Sep 11 13:38:13.104329 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.104307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.106724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.106707    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2a5fb372\""
Sep 11 13:38:13.116157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.116133    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:13.143569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.143524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb5gc\" (UniqueName: \"kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.143746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.143641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.244844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.244740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb5gc\" (UniqueName: \"kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.245006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.244879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.245868 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.245836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.256318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.256287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb5gc\" (UniqueName: \"kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc\") pod \"payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") " pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.414869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.414823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:13.693098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:13.693000    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:13.695611 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:38:13.695583    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274ce07f_8653_411e_8bda_8054a5b29464.slice/crio-bcb4b35345285c53632daf6ae58718b6a6a52f416c1e65723bdba2408cc61855 WatchSource:0}: Error finding container bcb4b35345285c53632daf6ae58718b6a6a52f416c1e65723bdba2408cc61855: Status 404 returned error can't find the container with id bcb4b35345285c53632daf6ae58718b6a6a52f416c1e65723bdba2408cc61855
Sep 11 13:38:14.210217 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.210185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:14.361388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.361357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" event={"ID":"274ce07f-8653-411e-8bda-8054a5b29464","Type":"ContainerStarted","Data":"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"}
Sep 11 13:38:14.361586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.361413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" event={"ID":"274ce07f-8653-411e-8bda-8054a5b29464","Type":"ContainerStarted","Data":"bcb4b35345285c53632daf6ae58718b6a6a52f416c1e65723bdba2408cc61855"}
Sep 11 13:38:14.361586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.361440    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:14.381314 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.381265    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" podStartSLOduration=1.381243555 podStartE2EDuration="1.381243555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:38:14.37985599 +0000 UTC m=+1018.773543428" watchObservedRunningTime="2026-09-11 13:38:14.381243555 +0000 UTC m=+1018.774930967"
Sep 11 13:38:14.466140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.466059    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:38:14.466290 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.466266    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy" containerID="cri-o://ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c" gracePeriod=30
Sep 11 13:38:14.479729 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.479702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:14.925805 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:14.925775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:15.206783 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.206706    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:38:15.207041 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.207028    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:15.207041 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.207041    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="78f726be-6435-40d2-bbbb-1a93c07c4658" containerName="istio-proxy"
Sep 11 13:38:15.210160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.210141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.212808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.212789    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-5a0fa1-openshift-default-dockercfg-6kt9d\""
Sep 11 13:38:15.224967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.224938    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:38:15.230971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.230947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:15.262896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.262853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.262900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.262977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lswzq\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263200    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.263280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.263233    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364155 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364120    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lswzq\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364612    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364824 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364868    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.364985 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.364952    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.365180 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.365154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.366900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.366869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.366994 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.366946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.374836 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.374808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.375172 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.375152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lswzq\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq\") pod \"e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") " pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.398770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.398737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:15.519943 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.519918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:15.795258 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.795227    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:38:15.797641 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:38:15.797615    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7738e8f_cdfa_4a81_b57d_cefa867b83d3.slice/crio-848f750355eedf946c44a915488012c5c6a30810582bc7769c0677fc07de2005 WatchSource:0}: Error finding container 848f750355eedf946c44a915488012c5c6a30810582bc7769c0677fc07de2005: Status 404 returned error can't find the container with id 848f750355eedf946c44a915488012c5c6a30810582bc7769c0677fc07de2005
Sep 11 13:38:15.799937 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.799904    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:15.799993 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.799971    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:15.800027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:15.799999    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:16.374357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:16.374265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerStarted","Data":"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"}
Sep 11 13:38:16.374357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:16.374323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerStarted","Data":"848f750355eedf946c44a915488012c5c6a30810582bc7769c0677fc07de2005"}
Sep 11 13:38:16.403765 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:16.403713    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podStartSLOduration=1.403695221 podStartE2EDuration="1.403695221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:38:16.401947989 +0000 UTC m=+1020.795635412" watchObservedRunningTime="2026-09-11 13:38:16.403695221 +0000 UTC m=+1020.797382631"
Sep 11 13:38:16.522341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:16.522305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:16.528044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:16.528012    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:17.378379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:17.378344    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:17.379432 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:17.379411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:18.369718 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:18.369645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:18.466814 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:18.466770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:19.532418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.532358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:19.720898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.720872    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:38:19.803646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.803843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803682    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.803843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803701    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q94rh\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.803843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803801    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803848    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803906    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.803981    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo\") pod \"31f269c6-c533-4037-a224-8e2b2afb903a\" (UID: \"31f269c6-c533-4037-a224-8e2b2afb903a\") "
Sep 11 13:38:19.804194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804072    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:19.804194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:19.804194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804171    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804210    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data" (OuterVolumeSpecName: "istio-data") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804405    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804420    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804429    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804438    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.804456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.804447    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/31f269c6-c533-4037-a224-8e2b2afb903a-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.806194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.806169    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:19.806291 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.806249    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:38:19.806394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.806377    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh" (OuterVolumeSpecName: "kube-api-access-q94rh") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "kube-api-access-q94rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:19.806459 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.806423    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token" (OuterVolumeSpecName: "istio-token") pod "31f269c6-c533-4037-a224-8e2b2afb903a" (UID: "31f269c6-c533-4037-a224-8e2b2afb903a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:19.904998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.904958    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.904998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.904991    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q94rh\" (UniqueName: \"kubernetes.io/projected/31f269c6-c533-4037-a224-8e2b2afb903a-kube-api-access-q94rh\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.904998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.905005    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/31f269c6-c533-4037-a224-8e2b2afb903a-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:19.905244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:19.905017    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/31f269c6-c533-4037-a224-8e2b2afb903a-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:20.397778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.397747    2576 generic.go:356] "Generic (PLEG): container finished" podID="31f269c6-c533-4037-a224-8e2b2afb903a" containerID="ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c" exitCode=0
Sep 11 13:38:20.397982 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.397799    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"
Sep 11 13:38:20.397982 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.397828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerDied","Data":"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"}
Sep 11 13:38:20.397982 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.397891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt" event={"ID":"31f269c6-c533-4037-a224-8e2b2afb903a","Type":"ContainerDied","Data":"1a81f6cf238f0537c1e7fd15835d391ab52125ce0f4f24feb944bb0d7076b016"}
Sep 11 13:38:20.397982 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.397920    2576 scope.go:117] "RemoveContainer" containerID="ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"
Sep 11 13:38:20.406157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.406143    2576 scope.go:117] "RemoveContainer" containerID="c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"
Sep 11 13:38:20.413935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.413919    2576 scope.go:117] "RemoveContainer" containerID="ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"
Sep 11 13:38:20.414204 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:20.414187    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c\": container with ID starting with ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c not found: ID does not exist" containerID="ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"
Sep 11 13:38:20.414260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.414215    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c"} err="failed to get container status \"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c\": rpc error: code = NotFound desc = could not find container \"ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c\": container with ID starting with ac8325bd719342ca74ca16f5c7865708557686e4047b5e0c8d6331db9440d22c not found: ID does not exist"
Sep 11 13:38:20.414260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.414232    2576 scope.go:117] "RemoveContainer" containerID="c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"
Sep 11 13:38:20.414464 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:20.414447    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e\": container with ID starting with c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e not found: ID does not exist" containerID="c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"
Sep 11 13:38:20.414546 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.414473    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e"} err="failed to get container status \"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e\": rpc error: code = NotFound desc = could not find container \"c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e\": container with ID starting with c1de87fa0c4089987ebdee2302d31f178f8bcdcfd6d104a67ebe671d72dbba2e not found: ID does not exist"
Sep 11 13:38:20.427594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.427571    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:38:20.436250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.436228    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-f9f26e-openshift-default-659995bc74-sp5qt"]
Sep 11 13:38:20.689677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:20.689644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:21.491895 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:21.491863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:22.248906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:22.248874    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" path="/var/lib/kubelet/pods/31f269c6-c533-4037-a224-8e2b2afb903a/volumes"
Sep 11 13:38:22.718411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:22.718379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:23.793748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:23.793716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:23.844467 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:23.844429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:24.039312 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.039265    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:24.429419 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.429375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:24.530113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.530048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:24.579109 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579076    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:38:24.579414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579402    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.579414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579415    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.579587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579428    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.579587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579433    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.579587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579482    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.579587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.579490    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="31f269c6-c533-4037-a224-8e2b2afb903a" containerName="istio-proxy"
Sep 11 13:38:24.584007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.583987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.586433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.586400    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-5a0fa1\""
Sep 11 13:38:24.592797 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.592774    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:38:24.654850 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.654812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.655019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.654864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7kv\" (UniqueName: \"kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.755865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.755761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.755865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.755838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7kv\" (UniqueName: \"kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.756545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.756525    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.763981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.763950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7kv\" (UniqueName: \"kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv\") pod \"payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:24.894549 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:24.894505    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:25.170309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.170283    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:38:25.172420 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:38:25.172388    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode23182e6_381d_4691_8553_d11f1393e94f.slice/crio-c30c769546d177c81ecf7e5034e42ad2e997558c2520a63af86a9f8984292f67 WatchSource:0}: Error finding container c30c769546d177c81ecf7e5034e42ad2e997558c2520a63af86a9f8984292f67: Status 404 returned error can't find the container with id c30c769546d177c81ecf7e5034e42ad2e997558c2520a63af86a9f8984292f67
Sep 11 13:38:25.369629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.369595    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:25.425104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.425009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" event={"ID":"e23182e6-381d-4691-8553-d11f1393e94f","Type":"ContainerStarted","Data":"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"}
Sep 11 13:38:25.425104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.425076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" event={"ID":"e23182e6-381d-4691-8553-d11f1393e94f","Type":"ContainerStarted","Data":"c30c769546d177c81ecf7e5034e42ad2e997558c2520a63af86a9f8984292f67"}
Sep 11 13:38:25.425308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.425107    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:25.441641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:25.441571    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" podStartSLOduration=1.44155104 podStartE2EDuration="1.44155104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:38:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:38:25.440369202 +0000 UTC m=+1029.834056615" watchObservedRunningTime="2026-09-11 13:38:25.44155104 +0000 UTC m=+1029.835238454"
Sep 11 13:38:33.668971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:33.668927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:35.516239 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:35.516186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:36.435061 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:36.435026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:36.435726 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:36.435657    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:38:37.543368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.543337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:37.582726 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.582694    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"]
Sep 11 13:38:37.587176 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.587147    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:37.589578 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.589553    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:38:37.589733 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.589618    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:38:37.598906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.598880    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"]
Sep 11 13:38:37.681280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.681246    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw\" (UID: \"903ff0f1-b7d0-41f6-b564-6f8a2c231639\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:37.781949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.781907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw\" (UID: \"903ff0f1-b7d0-41f6-b564-6f8a2c231639\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:37.782591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.782570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw\" (UID: \"903ff0f1-b7d0-41f6-b564-6f8a2c231639\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:37.898523 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:37.898414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:38.188603 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:38.188563    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"]
Sep 11 13:38:38.190821 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:38:38.190791    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod903ff0f1_b7d0_41f6_b564_6f8a2c231639.slice/crio-14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d WatchSource:0}: Error finding container 14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d: Status 404 returned error can't find the container with id 14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d
Sep 11 13:38:38.486802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:38.486712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw" event={"ID":"903ff0f1-b7d0-41f6-b564-6f8a2c231639","Type":"ContainerStarted","Data":"14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d"}
Sep 11 13:38:38.552375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:38.552341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:39.492917 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:39.492826    2576 generic.go:356] "Generic (PLEG): container finished" podID="903ff0f1-b7d0-41f6-b564-6f8a2c231639" containerID="5746afb8971eb0f3c6c1ee8155b47db9bce9cee8620b58115109faa3094e054e" exitCode=0
Sep 11 13:38:39.492917 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:39.492906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw" event={"ID":"903ff0f1-b7d0-41f6-b564-6f8a2c231639","Type":"ContainerDied","Data":"5746afb8971eb0f3c6c1ee8155b47db9bce9cee8620b58115109faa3094e054e"}
Sep 11 13:38:39.778682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:39.778640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:40.683175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:40.683141    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:40.829864 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:40.829828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca\") pod \"903ff0f1-b7d0-41f6-b564-6f8a2c231639\" (UID: \"903ff0f1-b7d0-41f6-b564-6f8a2c231639\") "
Sep 11 13:38:40.830356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:40.830231    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "903ff0f1-b7d0-41f6-b564-6f8a2c231639" (UID: "903ff0f1-b7d0-41f6-b564-6f8a2c231639"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:40.890139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:40.890107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:40.930782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:40.930747    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/903ff0f1-b7d0-41f6-b564-6f8a2c231639-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:41.503693 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:41.503661    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"
Sep 11 13:38:41.503832 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:41.503701    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw" event={"ID":"903ff0f1-b7d0-41f6-b564-6f8a2c231639","Type":"ContainerDied","Data":"14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d"}
Sep 11 13:38:41.503832 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:41.503747    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14bdb04b6f7ad0695806ab342afde78a00fe4fd12702bd2e763edcff931e069d"
Sep 11 13:38:42.188483 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:42.188427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:43.282764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:43.282734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:43.381307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:43.381233    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:44.439537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.439410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:44.520788 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.520756    2576 generic.go:356] "Generic (PLEG): container finished" podID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerID="22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89" exitCode=0
Sep 11 13:38:44.520955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.520829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerDied","Data":"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"}
Sep 11 13:38:44.521277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.521262    2576 scope.go:117] "RemoveContainer" containerID="22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"
Sep 11 13:38:44.522328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.522299    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:44.522398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.522366    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:44.522434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:44.522408    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:38:45.307669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:45.307633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:45.528349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:45.528313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerStarted","Data":"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"}
Sep 11 13:38:46.188776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:46.188745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:46.520684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:46.520650    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:46.522336 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:46.522304    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:46.522634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:46.522604    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:47.070650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:47.070598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:47.378396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:47.378294    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:47.520763 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:47.520724    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:47.521116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:47.521085    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:48.376649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:48.376616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:48.520567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:48.520520    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:48.520783 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:48.520605    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:49.066615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.066580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.114458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.114416    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:49.117931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.114704    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" podUID="274ce07f-8653-411e-8bda-8054a5b29464" containerName="payload-processing" containerID="cri-o://27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f" gracePeriod=30
Sep 11 13:38:49.155596 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.155567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.226151 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.226051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.315756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.315628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.356837 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.356721    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"]
Sep 11 13:38:49.359803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.359774    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a5fb372-6nqhw"]
Sep 11 13:38:49.395047 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.395002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.438218 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.438194    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:49.509800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.509766    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wb5gc\" (UniqueName: \"kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc\") pod \"274ce07f-8653-411e-8bda-8054a5b29464\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") "
Sep 11 13:38:49.510589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.510565    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume\") pod \"274ce07f-8653-411e-8bda-8054a5b29464\" (UID: \"274ce07f-8653-411e-8bda-8054a5b29464\") "
Sep 11 13:38:49.511332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.511309    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "274ce07f-8653-411e-8bda-8054a5b29464" (UID: "274ce07f-8653-411e-8bda-8054a5b29464"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:49.522886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.521890    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:49.522886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.521956    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:49.524353 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.524317    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc" (OuterVolumeSpecName: "kube-api-access-wb5gc") pod "274ce07f-8653-411e-8bda-8054a5b29464" (UID: "274ce07f-8653-411e-8bda-8054a5b29464"). InnerVolumeSpecName "kube-api-access-wb5gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:49.532931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.532902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:49.549380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.549351    2576 generic.go:356] "Generic (PLEG): container finished" podID="274ce07f-8653-411e-8bda-8054a5b29464" containerID="27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f" exitCode=0
Sep 11 13:38:49.549556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.549405    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"
Sep 11 13:38:49.549556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.549424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" event={"ID":"274ce07f-8653-411e-8bda-8054a5b29464","Type":"ContainerDied","Data":"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"}
Sep 11 13:38:49.549556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.549515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l" event={"ID":"274ce07f-8653-411e-8bda-8054a5b29464","Type":"ContainerDied","Data":"bcb4b35345285c53632daf6ae58718b6a6a52f416c1e65723bdba2408cc61855"}
Sep 11 13:38:49.549556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.549545    2576 scope.go:117] "RemoveContainer" containerID="27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"
Sep 11 13:38:49.558654 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.558634    2576 scope.go:117] "RemoveContainer" containerID="27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"
Sep 11 13:38:49.558916 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:38:49.558896    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f\": container with ID starting with 27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f not found: ID does not exist" containerID="27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"
Sep 11 13:38:49.558956 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.558924    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f"} err="failed to get container status \"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f\": rpc error: code = NotFound desc = could not find container \"27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f\": container with ID starting with 27ca9e3a6007c5317439d2c5330dc508f91dcbd7b910948af835a40a5ed75d6f not found: ID does not exist"
Sep 11 13:38:49.570951 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.570917    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:49.578732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.578712    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a5fb372-77668d9bbf-xtr8l"]
Sep 11 13:38:49.612344 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.612247    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/274ce07f-8653-411e-8bda-8054a5b29464-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:49.612589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.612569    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb5gc\" (UniqueName: \"kubernetes.io/projected/274ce07f-8653-411e-8bda-8054a5b29464-kube-api-access-wb5gc\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:38:49.887612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:49.887486    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:50.185373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.185289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:50.248576 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.248542    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274ce07f-8653-411e-8bda-8054a5b29464" path="/var/lib/kubelet/pods/274ce07f-8653-411e-8bda-8054a5b29464/volumes"
Sep 11 13:38:50.248881 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.248868    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="903ff0f1-b7d0-41f6-b564-6f8a2c231639" path="/var/lib/kubelet/pods/903ff0f1-b7d0-41f6-b564-6f8a2c231639/volumes"
Sep 11 13:38:50.329838 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.329804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:50.452347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.452255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:50.521653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.521614    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:50.521945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:50.521914    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:51.158728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:51.158690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:51.521462 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:51.521423    2576 patch_prober.go:28] interesting pod/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body=
Sep 11 13:38:51.522083 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:51.522050    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused"
Sep 11 13:38:51.939239 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:51.939159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:52.524669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:52.524635    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:52.525686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:52.525666    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:38:53.015137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:53.015109    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:54.051748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:54.051715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:54.919521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:54.919484    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:56.186857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:56.186818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:58.394671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:58.394640    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:58.460468 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:58.460439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:58.553919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:58.553876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:58.652682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:58.652586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:58.757352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:58.757311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:38:59.231730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:38:59.231690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:09.587567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:09.587423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:25.122780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:25.122745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:25.743895 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:25.743862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:31.917813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:31.917776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:31.975352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:31.975321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:32.074123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:32.074063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:32.162361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:32.162331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:48.042620 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:48.042587    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:54.395068 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:54.395038    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:54.452633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:54.452597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:54.542007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:54.541973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:54.636994 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:54.636959    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:56.848687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:56.848656    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:39:56.942210 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:39:56.942181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:02.407766 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.407732    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:02.408136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408055    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="903ff0f1-b7d0-41f6-b564-6f8a2c231639" containerName="revoke-keys"
Sep 11 13:40:02.408136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408066    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="903ff0f1-b7d0-41f6-b564-6f8a2c231639" containerName="revoke-keys"
Sep 11 13:40:02.408136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408080    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274ce07f-8653-411e-8bda-8054a5b29464" containerName="payload-processing"
Sep 11 13:40:02.408136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408086    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="274ce07f-8653-411e-8bda-8054a5b29464" containerName="payload-processing"
Sep 11 13:40:02.408266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408142    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="274ce07f-8653-411e-8bda-8054a5b29464" containerName="payload-processing"
Sep 11 13:40:02.408266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.408151    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="903ff0f1-b7d0-41f6-b564-6f8a2c231639" containerName="revoke-keys"
Sep 11 13:40:02.410461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.410444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.415433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.415386    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-5a0fa1\"/\"kube-root-ca.crt\""
Sep 11 13:40:02.415433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.415428    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-5a0fa1\"/\"default-dockercfg-wq5nh\""
Sep 11 13:40:02.416109 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.416085    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-5a0fa1\"/\"explicit-model-5a0fa1-kserve-self-signed-certs\""
Sep 11 13:40:02.416109 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.416096    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-5a0fa1\"/\"openshift-service-ca.crt\""
Sep 11 13:40:02.431557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.431527    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:02.443416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.443552 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfnpg\" (UniqueName: \"kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.443602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443555    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.443602 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.443684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.443684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.443650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.484961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.484927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:02.544597 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfnpg\" (UniqueName: \"kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.544795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.544795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.544795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.544795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.544795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.544722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.545061 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.545012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.545061 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.545021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.545158 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.545134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.547013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.546994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.547240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.547224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.569769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.569739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfnpg\" (UniqueName: \"kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg\") pod \"explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") " pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.719836 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.719725    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:02.998484 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:02.998412    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:03.001479 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:03.001454    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23d1f266_dfb5_4165_ada6_6248b7530ee1.slice/crio-e9514b2f7b4a39839a4b8f2c275ccdc3d028235b3e1d39438263805920f90690 WatchSource:0}: Error finding container e9514b2f7b4a39839a4b8f2c275ccdc3d028235b3e1d39438263805920f90690: Status 404 returned error can't find the container with id e9514b2f7b4a39839a4b8f2c275ccdc3d028235b3e1d39438263805920f90690
Sep 11 13:40:03.891489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:03.891454    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" event={"ID":"23d1f266-dfb5-4165-ada6-6248b7530ee1","Type":"ContainerStarted","Data":"4e91265f3f7d84424a644bfb1970c01e76b5ea857d385197e9c31a317fd3d93b"}
Sep 11 13:40:03.891489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:03.891511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" event={"ID":"23d1f266-dfb5-4165-ada6-6248b7530ee1","Type":"ContainerStarted","Data":"e9514b2f7b4a39839a4b8f2c275ccdc3d028235b3e1d39438263805920f90690"}
Sep 11 13:40:03.911511 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:03.911442    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" podStartSLOduration=1.911423282 podStartE2EDuration="1.911423282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:03.910918931 +0000 UTC m=+1128.304606340" watchObservedRunningTime="2026-09-11 13:40:03.911423282 +0000 UTC m=+1128.305110688"
Sep 11 13:40:05.185000 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:05.184971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:06.006490 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:06.006459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:09.643690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:09.643649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:12.720266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:12.720225    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:12.720266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:12.720275    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:12.733031 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:12.733005    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:12.943395 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:12.943367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:13.297795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:13.297763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:13.408481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:13.408447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:13.517291 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:13.517260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:13.625929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:13.625841    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:14.305879 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:14.305840    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:14.405865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:14.405835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:15.928695 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:15.928663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:22.692384 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:22.692332    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:22.795001 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:22.794965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:23.462128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:23.462097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:23.632038 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:23.631999    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.005786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.005753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.117921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.117885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.309847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.309821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.600669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.600584    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:40:30.603999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.603975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.607609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.607591    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1a269e4e-openshift-default-dockercfg-95mh5\""
Sep 11 13:40:30.626854 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.626823    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:40:30.664898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.664864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.702955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.702914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.702979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftkc7\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703223    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.703586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.703278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.803987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftkc7\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804126    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804161    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.804901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.804795    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.805056 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.805031    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.806755 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.806721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.806917 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.806900    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.815418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.815391    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.815560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.815541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftkc7\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7\") pod \"e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") " pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:30.843480 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.843446    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:30.914226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:30.914135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:31.245878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:31.245799    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:40:31.259376 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:31.259347    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e8ef20_76be_463f_a967_790cfcb21540.slice/crio-9b27533fb75932c5d1d18ff10c648ada143453a2b9a81e3cee6b2349ff7ac005 WatchSource:0}: Error finding container 9b27533fb75932c5d1d18ff10c648ada143453a2b9a81e3cee6b2349ff7ac005: Status 404 returned error can't find the container with id 9b27533fb75932c5d1d18ff10c648ada143453a2b9a81e3cee6b2349ff7ac005
Sep 11 13:40:31.261607 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:31.261573    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:31.261702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:31.261643    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:31.261702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:31.261671    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:32.023151 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:32.023110    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerStarted","Data":"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"}
Sep 11 13:40:32.023333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:32.023162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerStarted","Data":"9b27533fb75932c5d1d18ff10c648ada143453a2b9a81e3cee6b2349ff7ac005"}
Sep 11 13:40:32.057625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:32.057567    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podStartSLOduration=2.057549416 podStartE2EDuration="2.057549416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:32.055743395 +0000 UTC m=+1156.449430805" watchObservedRunningTime="2026-09-11 13:40:32.057549416 +0000 UTC m=+1156.451236826"
Sep 11 13:40:32.915001 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:32.914964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:32.921696 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:32.921662    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:33.027350 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:33.027318    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:33.028244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:33.028222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:40:35.869726 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.869690    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:35.950819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.950779    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:35.993748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.993713    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"]
Sep 11 13:40:35.997208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.997188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:35.999639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.999617    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:40:35.999831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:35.999813    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:40:36.012622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.012595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"]
Sep 11 13:40:36.051475 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.051446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm\" (UID: \"83977bc9-1f89-40b6-829a-6de3e5e3fe4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:36.152416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.152328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm\" (UID: \"83977bc9-1f89-40b6-829a-6de3e5e3fe4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:36.152991 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.152970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm\" (UID: \"83977bc9-1f89-40b6-829a-6de3e5e3fe4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:36.307694 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.307659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:36.345855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.345729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:36.581079 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:36.580952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"]
Sep 11 13:40:36.583565 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:36.583538    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83977bc9_1f89_40b6_829a_6de3e5e3fe4e.slice/crio-2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c WatchSource:0}: Error finding container 2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c: Status 404 returned error can't find the container with id 2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c
Sep 11 13:40:37.048065 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:37.048025    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm" event={"ID":"83977bc9-1f89-40b6-829a-6de3e5e3fe4e","Type":"ContainerStarted","Data":"1c9d6a291e4854a0740eccf436ba97717ae21231c5efc5612702c1cc24090e0a"}
Sep 11 13:40:37.048442 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:37.048088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm" event={"ID":"83977bc9-1f89-40b6-829a-6de3e5e3fe4e","Type":"ContainerStarted","Data":"2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c"}
Sep 11 13:40:37.069024 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:37.068970    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm" podStartSLOduration=2.06895464 podStartE2EDuration="2.06895464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:37.067256584 +0000 UTC m=+1161.460943997" watchObservedRunningTime="2026-09-11 13:40:37.06895464 +0000 UTC m=+1161.462642098"
Sep 11 13:40:37.402231 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:37.402194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:37.980087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:37.980056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:38.054776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:38.054737    2576 generic.go:356] "Generic (PLEG): container finished" podID="83977bc9-1f89-40b6-829a-6de3e5e3fe4e" containerID="1c9d6a291e4854a0740eccf436ba97717ae21231c5efc5612702c1cc24090e0a" exitCode=0
Sep 11 13:40:38.055157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:38.054812    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm" event={"ID":"83977bc9-1f89-40b6-829a-6de3e5e3fe4e","Type":"ContainerDied","Data":"1c9d6a291e4854a0740eccf436ba97717ae21231c5efc5612702c1cc24090e0a"}
Sep 11 13:40:38.161722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:38.161691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:38.747622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:38.747586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:39.186989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:39.186965    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:39.283325 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:39.283286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca\") pod \"83977bc9-1f89-40b6-829a-6de3e5e3fe4e\" (UID: \"83977bc9-1f89-40b6-829a-6de3e5e3fe4e\") "
Sep 11 13:40:39.283671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:39.283649    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "83977bc9-1f89-40b6-829a-6de3e5e3fe4e" (UID: "83977bc9-1f89-40b6-829a-6de3e5e3fe4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:40:39.384848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:39.384767    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/83977bc9-1f89-40b6-829a-6de3e5e3fe4e-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:39.757037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:39.756948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:40.064379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:40.064349    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"
Sep 11 13:40:40.064554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:40.064385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm" event={"ID":"83977bc9-1f89-40b6-829a-6de3e5e3fe4e","Type":"ContainerDied","Data":"2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c"}
Sep 11 13:40:40.064554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:40.064435    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2962af0cfff4d5dad3a5f7c1ad51c5f871a241bc9639ab205d6272619b20b17c"
Sep 11 13:40:41.099890 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:41.099859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:42.375878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:42.375847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:43.384513 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:43.384472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:43.509891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:43.509858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:43.660222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:43.660141    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:44.258716 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.258686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:44.440755 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.440718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:44.637038 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.637005    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:40:44.637414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.637402    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83977bc9-1f89-40b6-829a-6de3e5e3fe4e" containerName="revoke-keys"
Sep 11 13:40:44.637464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.637416    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="83977bc9-1f89-40b6-829a-6de3e5e3fe4e" containerName="revoke-keys"
Sep 11 13:40:44.637529 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.637469    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="83977bc9-1f89-40b6-829a-6de3e5e3fe4e" containerName="revoke-keys"
Sep 11 13:40:44.640013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.639997    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.642814 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.642791    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1a269e4e-dockercfg-dqxtb\""
Sep 11 13:40:44.643044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.643025    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1a269e4e\""
Sep 11 13:40:44.663078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.654465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:40:44.732641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.732606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.732641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.732652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8xh8\" (UniqueName: \"kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.826906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.826870    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:40:44.830308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.830291    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:44.833952 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.833925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.834082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.833976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8xh8\" (UniqueName: \"kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.834715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.834693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.849213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.849185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8xh8\" (UniqueName: \"kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8\") pod \"payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:44.851742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.851709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:40:44.934955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.934859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g55sb\" (UniqueName: \"kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:44.934955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.934901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:44.954671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:44.954642    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:45.036291 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.036251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g55sb\" (UniqueName: \"kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:45.036465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.036407    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:45.037381 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.037357    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:45.047591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.047563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g55sb\" (UniqueName: \"kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb\") pod \"payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") " pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:45.140782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.140743    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:45.147384 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.147352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:45.230042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.230009    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:40:45.233182 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:45.232916    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5467eb_0a42_43c4_9589_7c1a0328c87f.slice/crio-f59523e9011891bf3de4615ebb81c2a175142168d3df5bbcf2ae4598b0405f38 WatchSource:0}: Error finding container f59523e9011891bf3de4615ebb81c2a175142168d3df5bbcf2ae4598b0405f38: Status 404 returned error can't find the container with id f59523e9011891bf3de4615ebb81c2a175142168d3df5bbcf2ae4598b0405f38
Sep 11 13:40:45.425034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:45.425009    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:40:45.426676 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:45.426647    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f3535e_0e31_4301_abe7_6d6e0715f72f.slice/crio-cac7234220f9daffde94080bd389e2218d5026289ad97f7e4453ef0152394b0a WatchSource:0}: Error finding container cac7234220f9daffde94080bd389e2218d5026289ad97f7e4453ef0152394b0a: Status 404 returned error can't find the container with id cac7234220f9daffde94080bd389e2218d5026289ad97f7e4453ef0152394b0a
Sep 11 13:40:46.093679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.093638    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" event={"ID":"5b5467eb-0a42-43c4-9589-7c1a0328c87f","Type":"ContainerStarted","Data":"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"}
Sep 11 13:40:46.094198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.093711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" event={"ID":"5b5467eb-0a42-43c4-9589-7c1a0328c87f","Type":"ContainerStarted","Data":"f59523e9011891bf3de4615ebb81c2a175142168d3df5bbcf2ae4598b0405f38"}
Sep 11 13:40:46.094198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.093738    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:46.095350 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.095325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" event={"ID":"c0f3535e-0e31-4301-abe7-6d6e0715f72f","Type":"ContainerStarted","Data":"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"}
Sep 11 13:40:46.095481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.095371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" event={"ID":"c0f3535e-0e31-4301-abe7-6d6e0715f72f","Type":"ContainerStarted","Data":"cac7234220f9daffde94080bd389e2218d5026289ad97f7e4453ef0152394b0a"}
Sep 11 13:40:46.095481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.095399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:46.115283 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.115227    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" podStartSLOduration=2.115207895 podStartE2EDuration="2.115207895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:46.113324135 +0000 UTC m=+1170.507011553" watchObservedRunningTime="2026-09-11 13:40:46.115207895 +0000 UTC m=+1170.508895310"
Sep 11 13:40:46.135406 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.135345    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" podStartSLOduration=2.135323361 podStartE2EDuration="2.135323361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:46.134071198 +0000 UTC m=+1170.527758610" watchObservedRunningTime="2026-09-11 13:40:46.135323361 +0000 UTC m=+1170.529010773"
Sep 11 13:40:46.176921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:46.176878    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:47.182649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:47.182597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:48.023632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:48.023598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:49.072787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.072737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:49.754989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.754955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:49.788276 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.788246    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:40:49.788484 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.788442    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" podUID="e23182e6-381d-4691-8553-d11f1393e94f" containerName="payload-pre-processing" containerID="cri-o://d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38" gracePeriod=30
Sep 11 13:40:49.856719 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.856687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:49.985223 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.985031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:49.997585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:49.996804    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"]
Sep 11 13:40:50.001047 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.000999    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-5a0fa1-f4pmm"]
Sep 11 13:40:50.064634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.064607    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:40:50.098562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.098535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:50.116395 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.116361    2576 generic.go:356] "Generic (PLEG): container finished" podID="e23182e6-381d-4691-8553-d11f1393e94f" containerID="d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38" exitCode=0
Sep 11 13:40:50.116589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.116419    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"
Sep 11 13:40:50.116589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.116444    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" event={"ID":"e23182e6-381d-4691-8553-d11f1393e94f","Type":"ContainerDied","Data":"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"}
Sep 11 13:40:50.116589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.116518    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj" event={"ID":"e23182e6-381d-4691-8553-d11f1393e94f","Type":"ContainerDied","Data":"c30c769546d177c81ecf7e5034e42ad2e997558c2520a63af86a9f8984292f67"}
Sep 11 13:40:50.116589 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.116524    2576 scope.go:117] "RemoveContainer" containerID="d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"
Sep 11 13:40:50.124915 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.124889    2576 scope.go:117] "RemoveContainer" containerID="d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"
Sep 11 13:40:50.125209 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:40:50.125189    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38\": container with ID starting with d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38 not found: ID does not exist" containerID="d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"
Sep 11 13:40:50.125267 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.125218    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38"} err="failed to get container status \"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38\": rpc error: code = NotFound desc = could not find container \"d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38\": container with ID starting with d0c7d0e221279948eeaa37c5ea8e832dbb2a7c3d8c8faa429c62b4b7dd5a9c38 not found: ID does not exist"
Sep 11 13:40:50.190076 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.190039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m7kv\" (UniqueName: \"kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv\") pod \"e23182e6-381d-4691-8553-d11f1393e94f\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") "
Sep 11 13:40:50.190255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.190180    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume\") pod \"e23182e6-381d-4691-8553-d11f1393e94f\" (UID: \"e23182e6-381d-4691-8553-d11f1393e94f\") "
Sep 11 13:40:50.190599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.190573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e23182e6-381d-4691-8553-d11f1393e94f" (UID: "e23182e6-381d-4691-8553-d11f1393e94f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:40:50.192169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.192144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv" (OuterVolumeSpecName: "kube-api-access-8m7kv") pod "e23182e6-381d-4691-8553-d11f1393e94f" (UID: "e23182e6-381d-4691-8553-d11f1393e94f"). InnerVolumeSpecName "kube-api-access-8m7kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:40:50.249572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.249542    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83977bc9-1f89-40b6-829a-6de3e5e3fe4e" path="/var/lib/kubelet/pods/83977bc9-1f89-40b6-829a-6de3e5e3fe4e/volumes"
Sep 11 13:40:50.291351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.291262    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e23182e6-381d-4691-8553-d11f1393e94f-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:50.291351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.291292    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m7kv\" (UniqueName: \"kubernetes.io/projected/e23182e6-381d-4691-8553-d11f1393e94f-kube-api-access-8m7kv\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:50.386707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.386663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:50.431769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.431740    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:40:50.438630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.438601    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-5a0fa1-65d6fd6b74-sj4kj"]
Sep 11 13:40:50.553305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.553216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:50.888247 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:50.888160    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" podUID="23d1f266-dfb5-4165-ada6-6248b7530ee1" containerName="main" containerID="cri-o://4e91265f3f7d84424a644bfb1970c01e76b5ea857d385197e9c31a317fd3d93b" gracePeriod=45
Sep 11 13:40:51.124230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.124202    2576 generic.go:356] "Generic (PLEG): container finished" podID="23d1f266-dfb5-4165-ada6-6248b7530ee1" containerID="4e91265f3f7d84424a644bfb1970c01e76b5ea857d385197e9c31a317fd3d93b" exitCode=0
Sep 11 13:40:51.124569 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.124276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" event={"ID":"23d1f266-dfb5-4165-ada6-6248b7530ee1","Type":"ContainerDied","Data":"4e91265f3f7d84424a644bfb1970c01e76b5ea857d385197e9c31a317fd3d93b"}
Sep 11 13:40:51.143236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.143176    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:51.300060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300024    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300067    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfnpg\" (UniqueName: \"kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300130    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300153    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300216    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir\") pod \"23d1f266-dfb5-4165-ada6-6248b7530ee1\" (UID: \"23d1f266-dfb5-4165-ada6-6248b7530ee1\") "
Sep 11 13:40:51.300485 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300437    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache" (OuterVolumeSpecName: "model-cache") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:40:51.300610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300553    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home" (OuterVolumeSpecName: "home") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:40:51.300710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300623    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:40:51.300710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.300644    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-model-cache\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:51.302415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.302385    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm" (OuterVolumeSpecName: "dshm") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:40:51.302415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.302403    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg" (OuterVolumeSpecName: "kube-api-access-pfnpg") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "kube-api-access-pfnpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:40:51.302577 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.302420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23d1f266-dfb5-4165-ada6-6248b7530ee1" (UID: "23d1f266-dfb5-4165-ada6-6248b7530ee1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:40:51.394140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.394045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:51.401849 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.401820    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-home\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:51.401849 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.401850    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-tmp-dir\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:51.402078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.401864    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23d1f266-dfb5-4165-ada6-6248b7530ee1-dshm\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:51.402078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.401879    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfnpg\" (UniqueName: \"kubernetes.io/projected/23d1f266-dfb5-4165-ada6-6248b7530ee1-kube-api-access-pfnpg\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:51.402078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:51.401892    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23d1f266-dfb5-4165-ada6-6248b7530ee1-tls-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:40:52.131622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.131589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl" event={"ID":"23d1f266-dfb5-4165-ada6-6248b7530ee1","Type":"ContainerDied","Data":"e9514b2f7b4a39839a4b8f2c275ccdc3d028235b3e1d39438263805920f90690"}
Sep 11 13:40:52.132037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.131635    2576 scope.go:117] "RemoveContainer" containerID="4e91265f3f7d84424a644bfb1970c01e76b5ea857d385197e9c31a317fd3d93b"
Sep 11 13:40:52.132037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.131596    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"
Sep 11 13:40:52.168098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.168067    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:52.171999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.171974    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-5a0fa1/explicit-model-5a0fa1-kserve-7b56fc7865-tpnhl"]
Sep 11 13:40:52.248691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.248662    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23d1f266-dfb5-4165-ada6-6248b7530ee1" path="/var/lib/kubelet/pods/23d1f266-dfb5-4165-ada6-6248b7530ee1/volumes"
Sep 11 13:40:52.249020 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.249006    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e23182e6-381d-4691-8553-d11f1393e94f" path="/var/lib/kubelet/pods/e23182e6-381d-4691-8553-d11f1393e94f/volumes"
Sep 11 13:40:52.616069 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:52.616033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:53.543421 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:53.543387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:54.410429 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:54.410379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:55.458649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:55.458620    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:56.385185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:56.385152    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:57.066489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.066455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:57.102185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.102157    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:40:57.102349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.102227    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:40:57.183755 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.183721    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:57.298307 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.298271    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:40:57.298823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.298786    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy" containerID="cri-o://9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d" gracePeriod=30
Sep 11 13:40:57.341703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.341623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:57.688920 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.688818    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:57.852234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:57.852206    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:58.040878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.040849    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:40:58.041359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041341    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e23182e6-381d-4691-8553-d11f1393e94f" containerName="payload-pre-processing"
Sep 11 13:40:58.041411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041360    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e23182e6-381d-4691-8553-d11f1393e94f" containerName="payload-pre-processing"
Sep 11 13:40:58.041411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041377    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23d1f266-dfb5-4165-ada6-6248b7530ee1" containerName="main"
Sep 11 13:40:58.041411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041383    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="23d1f266-dfb5-4165-ada6-6248b7530ee1" containerName="main"
Sep 11 13:40:58.041521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041446    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e23182e6-381d-4691-8553-d11f1393e94f" containerName="payload-pre-processing"
Sep 11 13:40:58.041521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.041457    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="23d1f266-dfb5-4165-ada6-6248b7530ee1" containerName="main"
Sep 11 13:40:58.046010 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.045979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.049638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.049583    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-105163-openshift-default-dockercfg-2mdt8\""
Sep 11 13:40:58.056622 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.056587    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:40:58.109815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.109778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:58.167097 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167097 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167227    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mld9\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.167369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.167359    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268604 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268604 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268625    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mld9\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.268845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.268816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.269157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.269017    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.269217 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.269186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.269271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.269220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.269357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.269339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.269619 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.269598    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.271156 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.271139    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.271469 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.271449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.306568 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.292810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.306568 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.295264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mld9\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9\") pod \"e2e-noten-105163-openshift-default-85b4598d5b-fjx5q\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") " pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.313640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.313594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:58.357817 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.357772    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:58.481039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.480994    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:58.602188 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.602097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:58.645686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.645661    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:40:58.648253 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:40:58.648227    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb713dff4_90fe_46f0_b062_1838a0a807e5.slice/crio-e0c4ff99ea06efb82bdad33983ecb3d0826bf7d97bc3408ba006353e411c8341 WatchSource:0}: Error finding container e0c4ff99ea06efb82bdad33983ecb3d0826bf7d97bc3408ba006353e411c8341: Status 404 returned error can't find the container with id e0c4ff99ea06efb82bdad33983ecb3d0826bf7d97bc3408ba006353e411c8341
Sep 11 13:40:58.650368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.650336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:58.650425 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.650401    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:58.650458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.650434    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:40:58.890586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:58.890439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:59.170675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.170576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" event={"ID":"b713dff4-90fe-46f0-b062-1838a0a807e5","Type":"ContainerStarted","Data":"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"}
Sep 11 13:40:59.170675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.170635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" event={"ID":"b713dff4-90fe-46f0-b062-1838a0a807e5","Type":"ContainerStarted","Data":"e0c4ff99ea06efb82bdad33983ecb3d0826bf7d97bc3408ba006353e411c8341"}
Sep 11 13:40:59.198668 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.198617    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" podStartSLOduration=1.198601688 podStartE2EDuration="1.198601688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:40:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:40:59.196026169 +0000 UTC m=+1183.589713579" watchObservedRunningTime="2026-09-11 13:40:59.198601688 +0000 UTC m=+1183.592289100"
Sep 11 13:40:59.358554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.358491    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:59.364586 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.364561    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:40:59.492814 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.492726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:59.590063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.590027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:59.719197 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.719160    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:40:59.818060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:40:59.818031    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:00.175313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:00.175217    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:41:00.176088 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:00.176069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:41:01.108799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:01.108771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:01.913736 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:01.913706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:02.548521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.548479    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:41:02.714375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lswzq\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714389    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714437    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714474    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714530    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714600    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.714707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.714633    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs\") pod \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\" (UID: \"d7738e8f-cdfa-4a81-b57d-cefa867b83d3\") "
Sep 11 13:41:02.715195 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.715158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.715366 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.715340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.716374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.715963    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.716374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.716059    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:41:02.716374 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.716343    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.721728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.721677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:02.721991 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.721957    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:02.724566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.722901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq" (OuterVolumeSpecName: "kube-api-access-lswzq") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "kube-api-access-lswzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:02.730690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.729601    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7738e8f-cdfa-4a81-b57d-cefa867b83d3" (UID: "d7738e8f-cdfa-4a81-b57d-cefa867b83d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815514    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815557    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lswzq\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-kube-api-access-lswzq\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815572    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815586    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815599    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815614    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815626    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815639    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:02.815680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:02.815650    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7738e8f-cdfa-4a81-b57d-cefa867b83d3-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:03.191790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.191755    2576 generic.go:356] "Generic (PLEG): container finished" podID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerID="9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d" exitCode=0
Sep 11 13:41:03.192200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.191806    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"
Sep 11 13:41:03.192200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.191844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerDied","Data":"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"}
Sep 11 13:41:03.192200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.191899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp" event={"ID":"d7738e8f-cdfa-4a81-b57d-cefa867b83d3","Type":"ContainerDied","Data":"848f750355eedf946c44a915488012c5c6a30810582bc7769c0677fc07de2005"}
Sep 11 13:41:03.192200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.191929    2576 scope.go:117] "RemoveContainer" containerID="9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"
Sep 11 13:41:03.201618 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.201599    2576 scope.go:117] "RemoveContainer" containerID="22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"
Sep 11 13:41:03.215901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.215838    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:41:03.230194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.227657    2576 scope.go:117] "RemoveContainer" containerID="9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"
Sep 11 13:41:03.230194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.229940    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-5a0fa1-openshift-default-5b5cf6db44-hjhzp"]
Sep 11 13:41:03.231696 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:03.231657    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d\": container with ID starting with 9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d not found: ID does not exist" containerID="9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"
Sep 11 13:41:03.231849 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.231703    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d"} err="failed to get container status \"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d\": rpc error: code = NotFound desc = could not find container \"9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d\": container with ID starting with 9ceef90905f98d11852dd9557322bcdc679f032de20c23e8a6515b249648045d not found: ID does not exist"
Sep 11 13:41:03.231849 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.231732    2576 scope.go:117] "RemoveContainer" containerID="22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"
Sep 11 13:41:03.232158 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:03.232127    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89\": container with ID starting with 22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89 not found: ID does not exist" containerID="22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"
Sep 11 13:41:03.232272 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:03.232184    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89"} err="failed to get container status \"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89\": rpc error: code = NotFound desc = could not find container \"22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89\": container with ID starting with 22b34e4cb819c793f847136a94c791741c616283c5343b0dafce8cd5b90b5a89 not found: ID does not exist"
Sep 11 13:41:04.248169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:04.248133    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" path="/var/lib/kubelet/pods/d7738e8f-cdfa-4a81-b57d-cefa867b83d3/volumes"
Sep 11 13:41:05.152686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.152646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:05.436582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.436482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:05.535945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.535908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:05.806710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.806675    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"]
Sep 11 13:41:05.807457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.807432    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:05.807624 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.807610    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:05.807845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.807832    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:05.807937 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.807927    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:05.812823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.812794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:05.816673 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.816643    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:41:05.817439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.817419    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:41:05.830997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.830965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"]
Sep 11 13:41:05.945994 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:05.945962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk\" (UID: \"d6453024-870b-4878-a6f1-de1372febd25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:06.047205 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:06.047165    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk\" (UID: \"d6453024-870b-4878-a6f1-de1372febd25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:06.047881 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:06.047855    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk\" (UID: \"d6453024-870b-4878-a6f1-de1372febd25\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:06.132333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:06.132215    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:06.404789 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:06.404758    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"]
Sep 11 13:41:07.239933 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:07.239894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk" event={"ID":"d6453024-870b-4878-a6f1-de1372febd25","Type":"ContainerStarted","Data":"23dd5567c3b5d0b12bdfaca47a1de518f410650956b11afa105f55f6483e5898"}
Sep 11 13:41:07.240351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:07.239973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk" event={"ID":"d6453024-870b-4878-a6f1-de1372febd25","Type":"ContainerStarted","Data":"d40b0d4bdea14560108d3590db5eb74fc08088ab94e9b35b4a4b140cf322e692"}
Sep 11 13:41:08.247082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:08.247053    2576 generic.go:356] "Generic (PLEG): container finished" podID="d6453024-870b-4878-a6f1-de1372febd25" containerID="23dd5567c3b5d0b12bdfaca47a1de518f410650956b11afa105f55f6483e5898" exitCode=0
Sep 11 13:41:08.247829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:08.247806    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk" event={"ID":"d6453024-870b-4878-a6f1-de1372febd25","Type":"ContainerDied","Data":"23dd5567c3b5d0b12bdfaca47a1de518f410650956b11afa105f55f6483e5898"}
Sep 11 13:41:08.921615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:08.921559    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk" podStartSLOduration=3.921542433 podStartE2EDuration="3.921542433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:41:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:41:08.916287326 +0000 UTC m=+1193.309974738" watchObservedRunningTime="2026-09-11 13:41:08.921542433 +0000 UTC m=+1193.315229844"
Sep 11 13:41:09.393222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:09.393197    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:09.480909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:09.480871    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca\") pod \"d6453024-870b-4878-a6f1-de1372febd25\" (UID: \"d6453024-870b-4878-a6f1-de1372febd25\") "
Sep 11 13:41:09.481332 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:09.481303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6453024-870b-4878-a6f1-de1372febd25" (UID: "d6453024-870b-4878-a6f1-de1372febd25"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:41:09.581764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:09.581734    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6453024-870b-4878-a6f1-de1372febd25-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:09.924599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:09.924519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:10.259121 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:10.259089    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"
Sep 11 13:41:10.259302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:10.259115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk" event={"ID":"d6453024-870b-4878-a6f1-de1372febd25","Type":"ContainerDied","Data":"d40b0d4bdea14560108d3590db5eb74fc08088ab94e9b35b4a4b140cf322e692"}
Sep 11 13:41:10.259302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:10.259167    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d40b0d4bdea14560108d3590db5eb74fc08088ab94e9b35b4a4b140cf322e692"
Sep 11 13:41:10.844870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:10.844833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:11.879698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:11.879664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:12.276854 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.276821    2576 generic.go:356] "Generic (PLEG): container finished" podID="f7e8ef20-76be-463f-a967-790cfcb21540" containerID="34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21" exitCode=0
Sep 11 13:41:12.277033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.276892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerDied","Data":"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"}
Sep 11 13:41:12.277293 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.277275    2576 scope.go:117] "RemoveContainer" containerID="34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"
Sep 11 13:41:12.278270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.278244    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:12.278326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.278306    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:12.278361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.278336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:12.704315 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:12.704232    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:13.284287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:13.284251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerStarted","Data":"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"}
Sep 11 13:41:13.721932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:13.721820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:13.915063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:13.915022    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:13.916008 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:13.915978    2576 patch_prober.go:28] interesting pod/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 13:41:13.916131 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:13.916034    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 13:41:14.611255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:14.611214    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:14.915471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:14.914955    2576 patch_prober.go:28] interesting pod/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 13:41:14.915471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:14.915023    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 13:41:15.915361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:15.914919    2576 patch_prober.go:28] interesting pod/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 13:41:15.915361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:15.914982    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 13:41:16.915582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:16.915540    2576 patch_prober.go:28] interesting pod/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 13:41:16.916669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:16.916170    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 13:41:17.915512 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:17.915451    2576 patch_prober.go:28] interesting pod/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Sep 11 13:41:17.915823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:17.915789    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Sep 11 13:41:18.028127 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:18.028088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:18.918541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:18.918510    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:18.919373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:18.919353    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:20.338628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.338592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:20.354085 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.354057    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:41:20.354306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.354284    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" podUID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" containerName="payload-processing" containerID="cri-o://2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86" gracePeriod=30
Sep 11 13:41:20.359121 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.359095    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:41:20.359326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.359304    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" podUID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" containerName="payload-pre-processing" containerID="cri-o://942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d" gracePeriod=30
Sep 11 13:41:20.438315 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.438286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:20.588406 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.588349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:20.627857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.627832    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:41:20.658045 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.658020    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:41:20.685704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.685678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8xh8\" (UniqueName: \"kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8\") pod \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") "
Sep 11 13:41:20.685847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.685759    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume\") pod \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\" (UID: \"5b5467eb-0a42-43c4-9589-7c1a0328c87f\") "
Sep 11 13:41:20.685847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.685798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g55sb\" (UniqueName: \"kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb\") pod \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") "
Sep 11 13:41:20.685847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.685834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume\") pod \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\" (UID: \"c0f3535e-0e31-4301-abe7-6d6e0715f72f\") "
Sep 11 13:41:20.686190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.686154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b5467eb-0a42-43c4-9589-7c1a0328c87f" (UID: "5b5467eb-0a42-43c4-9589-7c1a0328c87f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:41:20.686305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.686239    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0f3535e-0e31-4301-abe7-6d6e0715f72f" (UID: "c0f3535e-0e31-4301-abe7-6d6e0715f72f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:41:20.688519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.688450    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8" (OuterVolumeSpecName: "kube-api-access-t8xh8") pod "5b5467eb-0a42-43c4-9589-7c1a0328c87f" (UID: "5b5467eb-0a42-43c4-9589-7c1a0328c87f"). InnerVolumeSpecName "kube-api-access-t8xh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:20.688884 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.688862    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb" (OuterVolumeSpecName: "kube-api-access-g55sb") pod "c0f3535e-0e31-4301-abe7-6d6e0715f72f" (UID: "c0f3535e-0e31-4301-abe7-6d6e0715f72f"). InnerVolumeSpecName "kube-api-access-g55sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:20.713943 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.713898    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"]
Sep 11 13:41:20.726656 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.726626    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-1a269e4e-5prbk"]
Sep 11 13:41:20.742956 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.742930    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:20.787522 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.787466    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8xh8\" (UniqueName: \"kubernetes.io/projected/5b5467eb-0a42-43c4-9589-7c1a0328c87f-kube-api-access-t8xh8\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:20.787522 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.787521    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b5467eb-0a42-43c4-9589-7c1a0328c87f-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:20.787739 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.787535    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g55sb\" (UniqueName: \"kubernetes.io/projected/c0f3535e-0e31-4301-abe7-6d6e0715f72f-kube-api-access-g55sb\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:20.787739 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:20.787544    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0f3535e-0e31-4301-abe7-6d6e0715f72f-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:21.048631 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.048602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:21.265033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.265006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:21.325458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.325371    2576 generic.go:356] "Generic (PLEG): container finished" podID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" containerID="2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86" exitCode=0
Sep 11 13:41:21.325860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.325838    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"
Sep 11 13:41:21.330775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.330740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" event={"ID":"c0f3535e-0e31-4301-abe7-6d6e0715f72f","Type":"ContainerDied","Data":"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"}
Sep 11 13:41:21.330905 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.330823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg" event={"ID":"c0f3535e-0e31-4301-abe7-6d6e0715f72f","Type":"ContainerDied","Data":"cac7234220f9daffde94080bd389e2218d5026289ad97f7e4453ef0152394b0a"}
Sep 11 13:41:21.330905 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.330855    2576 scope.go:117] "RemoveContainer" containerID="2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"
Sep 11 13:41:21.340185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.340141    2576 generic.go:356] "Generic (PLEG): container finished" podID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" containerID="942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d" exitCode=0
Sep 11 13:41:21.340556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.340225    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"
Sep 11 13:41:21.340556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.340238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" event={"ID":"5b5467eb-0a42-43c4-9589-7c1a0328c87f","Type":"ContainerDied","Data":"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"}
Sep 11 13:41:21.340556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.340329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt" event={"ID":"5b5467eb-0a42-43c4-9589-7c1a0328c87f","Type":"ContainerDied","Data":"f59523e9011891bf3de4615ebb81c2a175142168d3df5bbcf2ae4598b0405f38"}
Sep 11 13:41:21.345281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.345210    2576 scope.go:117] "RemoveContainer" containerID="2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"
Sep 11 13:41:21.345698 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:21.345668    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86\": container with ID starting with 2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86 not found: ID does not exist" containerID="2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"
Sep 11 13:41:21.345779 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.345714    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86"} err="failed to get container status \"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86\": rpc error: code = NotFound desc = could not find container \"2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86\": container with ID starting with 2a682101fdce5aa2ae8beea8ea87110c4c842bd726a70e31f82da36453c04c86 not found: ID does not exist"
Sep 11 13:41:21.345779 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.345755    2576 scope.go:117] "RemoveContainer" containerID="942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"
Sep 11 13:41:21.356057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.355890    2576 scope.go:117] "RemoveContainer" containerID="942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"
Sep 11 13:41:21.356171 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:21.356153    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d\": container with ID starting with 942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d not found: ID does not exist" containerID="942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"
Sep 11 13:41:21.356215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.356178    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d"} err="failed to get container status \"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d\": rpc error: code = NotFound desc = could not find container \"942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d\": container with ID starting with 942a08df12291cafa507d6774a90a8c930918c17a43f50356007258a3856198d not found: ID does not exist"
Sep 11 13:41:21.362082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.362061    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:41:21.366396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.366375    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1a269e4e-f55cf5db9-9kwsg"]
Sep 11 13:41:21.375818 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.375798    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:41:21.382435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:21.382403    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1a269e4e-5cffc9f5c4-xzktt"]
Sep 11 13:41:22.118757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:22.118725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:22.248608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:22.248575    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" path="/var/lib/kubelet/pods/5b5467eb-0a42-43c4-9589-7c1a0328c87f/volumes"
Sep 11 13:41:22.248946 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:22.248933    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" path="/var/lib/kubelet/pods/c0f3535e-0e31-4301-abe7-6d6e0715f72f/volumes"
Sep 11 13:41:22.249226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:22.249214    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6453024-870b-4878-a6f1-de1372febd25" path="/var/lib/kubelet/pods/d6453024-870b-4878-a6f1-de1372febd25/volumes"
Sep 11 13:41:23.278783 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:23.278746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:24.099968 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:24.099939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:25.128653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:25.128619    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:27.227250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.227216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:27.327685 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.327654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:27.449850 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.449823    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:41:27.450037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.450008    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy" containerID="cri-o://78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73" gracePeriod=30
Sep 11 13:41:27.472295 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.472239    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:27.809757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.809727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:27.908252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:27.908213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:28.038904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:28.038873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:31.565995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:31.565965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:32.694396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.694375    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:32.801333 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801303    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801417    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801473    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftkc7\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801533    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801564    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801592    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert\") pod \"f7e8ef20-76be-463f-a967-790cfcb21540\" (UID: \"f7e8ef20-76be-463f-a967-790cfcb21540\") "
Sep 11 13:41:32.801853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801778    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:32.801853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801791    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data" (OuterVolumeSpecName: "istio-data") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:32.802116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801966    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.802116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801985    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.802116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.801986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:32.802116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.802057    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:41:32.802320 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.802170    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:32.803708 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.803680    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:41:32.803799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.803708    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7" (OuterVolumeSpecName: "kube-api-access-ftkc7") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "kube-api-access-ftkc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:32.803995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.803973    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:41:32.803995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.803977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token" (OuterVolumeSpecName: "istio-token") pod "f7e8ef20-76be-463f-a967-790cfcb21540" (UID: "f7e8ef20-76be-463f-a967-790cfcb21540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:41:32.903152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903107    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903144    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7e8ef20-76be-463f-a967-790cfcb21540-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903159    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftkc7\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-kube-api-access-ftkc7\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903175    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903189    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7e8ef20-76be-463f-a967-790cfcb21540-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903202    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7e8ef20-76be-463f-a967-790cfcb21540-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:32.903458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:32.903216    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7e8ef20-76be-463f-a967-790cfcb21540-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:41:33.399467 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.399435    2576 generic.go:356] "Generic (PLEG): container finished" podID="f7e8ef20-76be-463f-a967-790cfcb21540" containerID="78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73" exitCode=0
Sep 11 13:41:33.399680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.399526    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"
Sep 11 13:41:33.399680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.399524    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerDied","Data":"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"}
Sep 11 13:41:33.399680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.399648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw" event={"ID":"f7e8ef20-76be-463f-a967-790cfcb21540","Type":"ContainerDied","Data":"9b27533fb75932c5d1d18ff10c648ada143453a2b9a81e3cee6b2349ff7ac005"}
Sep 11 13:41:33.399680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.399671    2576 scope.go:117] "RemoveContainer" containerID="78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"
Sep 11 13:41:33.408658 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.408642    2576 scope.go:117] "RemoveContainer" containerID="34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"
Sep 11 13:41:33.416222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.416207    2576 scope.go:117] "RemoveContainer" containerID="78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"
Sep 11 13:41:33.416476 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:33.416458    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73\": container with ID starting with 78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73 not found: ID does not exist" containerID="78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"
Sep 11 13:41:33.416642 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.416484    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73"} err="failed to get container status \"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73\": rpc error: code = NotFound desc = could not find container \"78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73\": container with ID starting with 78fb24f2554981c85294cf72945fb4cb0c8186e78632f9f07f03097181a8bf73 not found: ID does not exist"
Sep 11 13:41:33.416642 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.416528    2576 scope.go:117] "RemoveContainer" containerID="34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"
Sep 11 13:41:33.416795 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:41:33.416778    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21\": container with ID starting with 34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21 not found: ID does not exist" containerID="34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"
Sep 11 13:41:33.416844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.416800    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21"} err="failed to get container status \"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21\": rpc error: code = NotFound desc = could not find container \"34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21\": container with ID starting with 34b0b19141716fe4817336ca815073afe34fdb23ebe47301d0d8eece10027b21 not found: ID does not exist"
Sep 11 13:41:33.428418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.428396    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:41:33.437617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:33.437597    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1a269e4e-openshift-default-7544bccd5b-lz8xw"]
Sep 11 13:41:34.248090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:34.248046    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" path="/var/lib/kubelet/pods/f7e8ef20-76be-463f-a967-790cfcb21540/volumes"
Sep 11 13:41:34.432645 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:34.432614    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:34.539286 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:34.539257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:34.634866 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:34.634836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:34.782851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:34.782820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:38.185471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:38.185440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:41.279477 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:41.279441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:41.383420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:41.383390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:42.190448 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.190414    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:42.462562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462466    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462776    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" containerName="payload-pre-processing"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462787    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" containerName="payload-pre-processing"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462799    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462805    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7738e8f-cdfa-4a81-b57d-cefa867b83d3" containerName="istio-proxy"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462820    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" containerName="payload-processing"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462826    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" containerName="payload-processing"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462836    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462841    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462847    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6453024-870b-4878-a6f1-de1372febd25" containerName="revoke-keys"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462853    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6453024-870b-4878-a6f1-de1372febd25" containerName="revoke-keys"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462858    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.462892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462863    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.463240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462912    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0f3535e-0e31-4301-abe7-6d6e0715f72f" containerName="payload-processing"
Sep 11 13:41:42.463240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462921    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.463240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462926    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7e8ef20-76be-463f-a967-790cfcb21540" containerName="istio-proxy"
Sep 11 13:41:42.463240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462934    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b5467eb-0a42-43c4-9589-7c1a0328c87f" containerName="payload-pre-processing"
Sep 11 13:41:42.463240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.462940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6453024-870b-4878-a6f1-de1372febd25" containerName="revoke-keys"
Sep 11 13:41:42.467154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.467126    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.470026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.470005    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-470eeb-openshift-default-dockercfg-nzpqz\""
Sep 11 13:41:42.488653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.488628    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:41:42.509383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.509352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:42.585051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8wqv\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585357    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.585597 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.585475    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686461    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8wqv\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686546    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686570    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.686748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686680    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.687100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.687100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.686949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.687100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.687041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.687100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.687070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.687436 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.687417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.688690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.688666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.688940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.688923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.694892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.694867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8wqv\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.694973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.694947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token\") pod \"e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") " pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.778411 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.778373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:42.823489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:42.823455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:43.059972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.059764    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:41:43.065061 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:41:43.065028    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9db282_ccda_49e3_a271_934d70aad7b6.slice/crio-4eebad051cfc893d27408110b443d23b70f3460b2b76d655b3025d6378300bc4 WatchSource:0}: Error finding container 4eebad051cfc893d27408110b443d23b70f3460b2b76d655b3025d6378300bc4: Status 404 returned error can't find the container with id 4eebad051cfc893d27408110b443d23b70f3460b2b76d655b3025d6378300bc4
Sep 11 13:41:43.067402 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.067366    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:43.067562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.067429    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:43.067562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.067460    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:41:43.452557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.452458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerStarted","Data":"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"}
Sep 11 13:41:43.452557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.452522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerStarted","Data":"4eebad051cfc893d27408110b443d23b70f3460b2b76d655b3025d6378300bc4"}
Sep 11 13:41:43.478036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.477987    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podStartSLOduration=1.477971365 podStartE2EDuration="1.477971365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:41:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:41:43.476966132 +0000 UTC m=+1227.870653544" watchObservedRunningTime="2026-09-11 13:41:43.477971365 +0000 UTC m=+1227.871658775"
Sep 11 13:41:43.778654 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:43.778618    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:44.784127 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:44.784089    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:44.798461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:44.798433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:45.449921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:45.449889    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:45.462409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:45.462381    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:45.463478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:45.463455    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:41:45.572093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:45.572065    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:45.844120 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:45.844085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:46.843043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:46.843007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:46.946093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:46.946062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:47.935446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:47.935417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:48.898634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:48.898605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:49.998433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:49.998372    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:50.066152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.066113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:50.165558 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.165523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:50.558850 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.558816    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:50.652675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.652633    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:41:50.751060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.751019    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:41:50.754554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.754533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.757033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.757008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-470eeb-dockercfg-28bcb\""
Sep 11 13:41:50.757033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.757020    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-470eeb\""
Sep 11 13:41:50.767019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.766998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:41:50.850326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.850249    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:41:50.853625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.853608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:50.868328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.868297    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:41:50.868562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.868543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln8h8\" (UniqueName: \"kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.868617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.868581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.969760 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.969721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:50.969949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.969790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln8h8\" (UniqueName: \"kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.969949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.969815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.969949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.969835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5zhx\" (UniqueName: \"kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:50.970461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.970442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:50.984343 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:50.984316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln8h8\" (UniqueName: \"kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8\") pod \"payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:51.065595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.065558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:51.070823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.070795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:51.070944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.070895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5zhx\" (UniqueName: \"kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:51.071455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.071438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:51.079049 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.079024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5zhx\" (UniqueName: \"kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx\") pod \"payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:51.163880 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.163842    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:51.337347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.337323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:41:51.339572 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:41:51.339544    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba205218_35c2_45d9_89f4_3a9318037148.slice/crio-0b932218a2ac87cd7cae1483575f9e0a4d30ac8641726c179146660312751ee7 WatchSource:0}: Error finding container 0b932218a2ac87cd7cae1483575f9e0a4d30ac8641726c179146660312751ee7: Status 404 returned error can't find the container with id 0b932218a2ac87cd7cae1483575f9e0a4d30ac8641726c179146660312751ee7
Sep 11 13:41:51.438843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.438819    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:41:51.440568 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:41:51.440530    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15f0cd5_785e_46bf_8905_e463b9e32535.slice/crio-c05c17399d3a48df6ae047817afb20677e830dda5a5118a73742aa7eefa732f4 WatchSource:0}: Error finding container c05c17399d3a48df6ae047817afb20677e830dda5a5118a73742aa7eefa732f4: Status 404 returned error can't find the container with id c05c17399d3a48df6ae047817afb20677e830dda5a5118a73742aa7eefa732f4
Sep 11 13:41:51.491853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.491822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" event={"ID":"c15f0cd5-785e-46bf-8905-e463b9e32535","Type":"ContainerStarted","Data":"c05c17399d3a48df6ae047817afb20677e830dda5a5118a73742aa7eefa732f4"}
Sep 11 13:41:51.493352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.493331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" event={"ID":"ba205218-35c2-45d9-89f4-3a9318037148","Type":"ContainerStarted","Data":"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"}
Sep 11 13:41:51.493414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.493371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" event={"ID":"ba205218-35c2-45d9-89f4-3a9318037148","Type":"ContainerStarted","Data":"0b932218a2ac87cd7cae1483575f9e0a4d30ac8641726c179146660312751ee7"}
Sep 11 13:41:51.493414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.493399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:41:51.516026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:51.515971    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" podStartSLOduration=1.51595116 podStartE2EDuration="1.51595116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:41:51.512947978 +0000 UTC m=+1235.906635387" watchObservedRunningTime="2026-09-11 13:41:51.51595116 +0000 UTC m=+1235.909638572"
Sep 11 13:41:52.499819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:52.499785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" event={"ID":"c15f0cd5-785e-46bf-8905-e463b9e32535","Type":"ContainerStarted","Data":"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"}
Sep 11 13:41:52.500304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:52.499925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:41:52.520066 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:41:52.520013    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" podStartSLOduration=2.519993085 podStartE2EDuration="2.519993085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:41:52.517367509 +0000 UTC m=+1236.911054933" watchObservedRunningTime="2026-09-11 13:41:52.519993085 +0000 UTC m=+1236.913680499"
Sep 11 13:42:02.502410 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:02.502373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:42:03.507700 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:03.507668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:42:06.512756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:06.512714    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:06.611208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:06.611169    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:06.852584 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:06.852556    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:42:06.852771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:06.852751    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" podUID="b713dff4-90fe-46f0-b062-1838a0a807e5" containerName="istio-proxy" containerID="cri-o://8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d" gracePeriod=30
Sep 11 13:42:06.865361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:06.865331    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:07.082990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:07.082905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:07.190563 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:07.190531    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:10.587896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.587858    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerID="3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016" exitCode=0
Sep 11 13:42:10.588311 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.587932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerDied","Data":"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"}
Sep 11 13:42:10.588357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.588348    2576 scope.go:117] "RemoveContainer" containerID="3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"
Sep 11 13:42:10.589508 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.589466    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:10.589572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.589557    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:10.589609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:10.589596    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:11.595070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:11.595035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerStarted","Data":"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"}
Sep 11 13:42:11.779027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:11.778988    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:42:11.780009 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:11.779984    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 13:42:11.780119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:11.780057    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 13:42:12.130815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.130720    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:42:12.163806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.163806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163894    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163937    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mld9\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.163969    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164000    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164022    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert\") pod \"b713dff4-90fe-46f0-b062-1838a0a807e5\" (UID: \"b713dff4-90fe-46f0-b062-1838a0a807e5\") "
Sep 11 13:42:12.164359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164077    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:12.164359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164168    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:12.164359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164191    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:12.164617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:12.164617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164394    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:42:12.164617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164427    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.164617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164448    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.164617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.164462    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.166400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.166337    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:42:12.166893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.166677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9" (OuterVolumeSpecName: "kube-api-access-6mld9") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "kube-api-access-6mld9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:12.166893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.166709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:42:12.167156 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.167124    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b713dff4-90fe-46f0-b062-1838a0a807e5" (UID: "b713dff4-90fe-46f0-b062-1838a0a807e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:42:12.265637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265610    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.265637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265635    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mld9\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-kube-api-access-6mld9\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.265825 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265646    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.265825 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265655    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b713dff4-90fe-46f0-b062-1838a0a807e5-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.265825 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265664    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b713dff4-90fe-46f0-b062-1838a0a807e5-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.265825 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.265672    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b713dff4-90fe-46f0-b062-1838a0a807e5-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:12.276931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.276895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:12.601282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.601253    2576 generic.go:356] "Generic (PLEG): container finished" podID="b713dff4-90fe-46f0-b062-1838a0a807e5" containerID="8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d" exitCode=0
Sep 11 13:42:12.601742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.601302    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"
Sep 11 13:42:12.601742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.601342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" event={"ID":"b713dff4-90fe-46f0-b062-1838a0a807e5","Type":"ContainerDied","Data":"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"}
Sep 11 13:42:12.601742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.601395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q" event={"ID":"b713dff4-90fe-46f0-b062-1838a0a807e5","Type":"ContainerDied","Data":"e0c4ff99ea06efb82bdad33983ecb3d0826bf7d97bc3408ba006353e411c8341"}
Sep 11 13:42:12.601742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.601417    2576 scope.go:117] "RemoveContainer" containerID="8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"
Sep 11 13:42:12.613905 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.613606    2576 scope.go:117] "RemoveContainer" containerID="8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"
Sep 11 13:42:12.614204 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:42:12.614119    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d\": container with ID starting with 8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d not found: ID does not exist" containerID="8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"
Sep 11 13:42:12.614204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.614154    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d"} err="failed to get container status \"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d\": rpc error: code = NotFound desc = could not find container \"8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d\": container with ID starting with 8e47b5184ab1d6cf0ece9a19f89bf1ef19af7c365b3bb8a8971270da50549b0d not found: ID does not exist"
Sep 11 13:42:12.616461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.616377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:12.624188 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.624164    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:42:12.633837 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.633809    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-105163-openshift-default-85b4598d5b-fjx5q"]
Sep 11 13:42:12.779742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.779709    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 13:42:12.779932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:12.779775    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 13:42:13.749054 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:13.749016    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:13.779136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:13.779096    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 13:42:13.779313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:13.779159    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 13:42:14.254455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:14.254419    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b713dff4-90fe-46f0-b062-1838a0a807e5" path="/var/lib/kubelet/pods/b713dff4-90fe-46f0-b062-1838a0a807e5/volumes"
Sep 11 13:42:14.605026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:14.604944    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:14.778868 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:14.778825    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 13:42:14.779340 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:14.778912    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 13:42:15.425363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.425320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:15.462959 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.462922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:42:15.615256 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.615212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:15.744137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.744050    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:15.779346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.779306    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 11 13:42:15.779751 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:15.779361    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 11 13:42:16.062613 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.062582    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:42:16.063014 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.062996    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b713dff4-90fe-46f0-b062-1838a0a807e5" containerName="istio-proxy"
Sep 11 13:42:16.063104 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.063017    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b713dff4-90fe-46f0-b062-1838a0a807e5" containerName="istio-proxy"
Sep 11 13:42:16.063157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.063123    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b713dff4-90fe-46f0-b062-1838a0a807e5" containerName="istio-proxy"
Sep 11 13:42:16.065201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.065180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.065696 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.065676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:16.068067 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.068051    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-e52aaa-openshift-default-dockercfg-nhr7f\""
Sep 11 13:42:16.083343 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.083319    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:42:16.201778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.201746    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.201891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.201790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wq7w\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.201942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.201887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.201989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.201929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.202037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.202028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.202074 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.202046    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.202074 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.202061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.202154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.202080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.202154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.202120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wq7w\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303277    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303770    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.303953 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.303896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.304264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.304247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.306483 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.306435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.314040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.313955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wq7w\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.314632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.314610    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.318375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.317704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy\") pod \"e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") " pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.374758 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.374723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:16.420525 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.418345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:16.671223 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.671110    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:42:16.674731 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:42:16.674697    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd91f2f_9307_4e50_9f51_615418672be6.slice/crio-2df4ec01a6b9bc0541e112ca7523f5d411da4d795bffc4f7c0fc1117c04d43b2 WatchSource:0}: Error finding container 2df4ec01a6b9bc0541e112ca7523f5d411da4d795bffc4f7c0fc1117c04d43b2: Status 404 returned error can't find the container with id 2df4ec01a6b9bc0541e112ca7523f5d411da4d795bffc4f7c0fc1117c04d43b2
Sep 11 13:42:16.677887 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.677842    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:16.677996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.677933    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:16.677996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.677973    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:16.781140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.781104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:16.980229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.980047    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:42:16.981928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:16.981894    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:42:17.634302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:17.634264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerStarted","Data":"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"}
Sep 11 13:42:17.634606 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:17.634324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerStarted","Data":"2df4ec01a6b9bc0541e112ca7523f5d411da4d795bffc4f7c0fc1117c04d43b2"}
Sep 11 13:42:17.659566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:17.659514    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podStartSLOduration=1.659483132 podStartE2EDuration="1.659483132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:17.658868536 +0000 UTC m=+1262.052555950" watchObservedRunningTime="2026-09-11 13:42:17.659483132 +0000 UTC m=+1262.053170542"
Sep 11 13:42:17.818896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:17.818865    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:18.375060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:18.375024    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:18.379966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:18.379942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:18.639537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:18.639442    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:18.640544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:18.640525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:19.309805 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:19.309772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:19.796953 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:19.796901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:19.932548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:19.932485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:20.066770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:20.066681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:21.195056 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:21.195028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:22.218826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:22.218792    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:22.658555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:22.658521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:23.401219 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:23.401189    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:24.218582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:24.218551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:25.000347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:25.000321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.165507 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.165464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.256280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.256242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.553710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.553678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.814690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.814605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.910781 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.910749    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:26.958643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.958604    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:42:26.961878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.961859    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:26.964186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.964165    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-e52aaa\""
Sep 11 13:42:26.964293 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.964209    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-e52aaa-dockercfg-sqg5d\""
Sep 11 13:42:26.977673 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:26.977642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:42:27.129829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.129727    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.129829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.129816    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdsg\" (UniqueName: \"kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.230823 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.230785    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.231236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.230849    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdsg\" (UniqueName: \"kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.231532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.231480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.239170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.239135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdsg\" (UniqueName: \"kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg\") pod \"payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.273100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.273063    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.322205 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.322176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:27.543809 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.543781    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:42:27.546056 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:42:27.546030    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51247622_b170_463f_91e1_3511906b3563.slice/crio-937ac5605b8f1ecae0a661d6684cabb2944dc772c5f53d027bfb10f30f7aea4f WatchSource:0}: Error finding container 937ac5605b8f1ecae0a661d6684cabb2944dc772c5f53d027bfb10f30f7aea4f: Status 404 returned error can't find the container with id 937ac5605b8f1ecae0a661d6684cabb2944dc772c5f53d027bfb10f30f7aea4f
Sep 11 13:42:27.685742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.685705    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" event={"ID":"51247622-b170-463f-91e1-3511906b3563","Type":"ContainerStarted","Data":"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"}
Sep 11 13:42:27.685942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.685762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" event={"ID":"51247622-b170-463f-91e1-3511906b3563","Type":"ContainerStarted","Data":"937ac5605b8f1ecae0a661d6684cabb2944dc772c5f53d027bfb10f30f7aea4f"}
Sep 11 13:42:27.685942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.685856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:27.708872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:27.708812    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" podStartSLOduration=1.7087900230000002 podStartE2EDuration="1.708790023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:27.706772755 +0000 UTC m=+1272.100460166" watchObservedRunningTime="2026-09-11 13:42:27.708790023 +0000 UTC m=+1272.102477434"
Sep 11 13:42:28.111457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:28.111427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:38.692838 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:38.692802    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:42:48.549462 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:48.549431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:48.897774 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:48.897696    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:49.782489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:49.782453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:49.881995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:49.881966    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:50.194222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:50.194116    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:50.777897 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:50.777862    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:51.599806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:51.599773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:52.583489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.583459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:52.801916 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.801882    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"]
Sep 11 13:42:52.805518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.805478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:52.808348 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.808318    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:42:52.808611 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.808593    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:42:52.814352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.814320    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"]
Sep 11 13:42:52.865464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.865375    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc\" (UID: \"aa6f915b-d1f9-4435-b283-5fdb39e23432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:52.966086 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.966048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc\" (UID: \"aa6f915b-d1f9-4435-b283-5fdb39e23432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:52.966720 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:52.966696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc\" (UID: \"aa6f915b-d1f9-4435-b283-5fdb39e23432\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:53.124001 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.123906    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:53.397756 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.397731    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"]
Sep 11 13:42:53.400068 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:42:53.400012    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6f915b_d1f9_4435_b283_5fdb39e23432.slice/crio-9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6 WatchSource:0}: Error finding container 9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6: Status 404 returned error can't find the container with id 9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6
Sep 11 13:42:53.700588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.700479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:53.809187 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.809155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc" event={"ID":"aa6f915b-d1f9-4435-b283-5fdb39e23432","Type":"ContainerStarted","Data":"34c298d03872a43ce66549b64ed8d8b5f4bae7f50c30235e2f73b54ecf0aac73"}
Sep 11 13:42:53.809590 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.809220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc" event={"ID":"aa6f915b-d1f9-4435-b283-5fdb39e23432","Type":"ContainerStarted","Data":"9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6"}
Sep 11 13:42:53.826183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:53.826133    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc" podStartSLOduration=1.826115624 podStartE2EDuration="1.826115624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:53.825127954 +0000 UTC m=+1298.218815366" watchObservedRunningTime="2026-09-11 13:42:53.826115624 +0000 UTC m=+1298.219803038"
Sep 11 13:42:54.493871 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:54.493769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:54.816207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:54.816176    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa6f915b-d1f9-4435-b283-5fdb39e23432" containerID="34c298d03872a43ce66549b64ed8d8b5f4bae7f50c30235e2f73b54ecf0aac73" exitCode=0
Sep 11 13:42:54.816677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:54.816268    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc" event={"ID":"aa6f915b-d1f9-4435-b283-5fdb39e23432","Type":"ContainerDied","Data":"34c298d03872a43ce66549b64ed8d8b5f4bae7f50c30235e2f73b54ecf0aac73"}
Sep 11 13:42:55.644475 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:55.644440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:55.971847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:55.971814    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:56.097745 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.097705    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca\") pod \"aa6f915b-d1f9-4435-b283-5fdb39e23432\" (UID: \"aa6f915b-d1f9-4435-b283-5fdb39e23432\") "
Sep 11 13:42:56.098087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.098059    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa6f915b-d1f9-4435-b283-5fdb39e23432" (UID: "aa6f915b-d1f9-4435-b283-5fdb39e23432"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:42:56.198875 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.198828    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa6f915b-d1f9-4435-b283-5fdb39e23432-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:42:56.536641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.536613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:56.828089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.828002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"
Sep 11 13:42:56.828089 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.828020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc" event={"ID":"aa6f915b-d1f9-4435-b283-5fdb39e23432","Type":"ContainerDied","Data":"9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6"}
Sep 11 13:42:56.828309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.828099    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9dfabbdcf03653b51d87a3d44675b6a411850576861f3c3ade4748e291c8d0f6"
Sep 11 13:42:56.829757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.829729    2576 generic.go:356] "Generic (PLEG): container finished" podID="8fd91f2f-9307-4e50-9f51-615418672be6" containerID="1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4" exitCode=0
Sep 11 13:42:56.829867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.829782    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerDied","Data":"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"}
Sep 11 13:42:56.830232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.830199    2576 scope.go:117] "RemoveContainer" containerID="1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"
Sep 11 13:42:56.831398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.831369    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:56.831469 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.831449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:56.831531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:56.831479    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:57.489356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:57.489326    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:57.836353 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:57.836321    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerStarted","Data":"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"}
Sep 11 13:42:58.374939 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.374907    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:42:58.375154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.375132    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:42:58.375201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.375181    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:42:58.487841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.487814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:58.598475 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.598446    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:58.873330 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.873292    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:42:58.873749 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.873732    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6f915b-d1f9-4435-b283-5fdb39e23432" containerName="revoke-keys"
Sep 11 13:42:58.873859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.873751    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6f915b-d1f9-4435-b283-5fdb39e23432" containerName="revoke-keys"
Sep 11 13:42:58.873859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.873854    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6f915b-d1f9-4435-b283-5fdb39e23432" containerName="revoke-keys"
Sep 11 13:42:58.875840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.875819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.879179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.879158    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c0f589fb-openshift-default-dockercfg-s4tlh\""
Sep 11 13:42:58.894193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.894171    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2g2l\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924376    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.924617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:58.930557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:58.925997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:59.025520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025520 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025489    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2g2l\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.025787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.026044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025839    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.026044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.026044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.025942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.029519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.026970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.029519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.027468    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.029519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.028071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.029519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.028163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.030057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.030029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.033280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.033254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.038357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.038324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2g2l\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.043998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.043978    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token\") pod \"e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") " pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.082289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.082248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:42:59.186313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.186226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:42:59.375966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.375933    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:42:59.376123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.375993    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:42:59.499790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.499765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:42:59.501172 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:42:59.501140    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cb895b_6ec1_4636_9ff1_a4b3bae05498.slice/crio-bc0e24dbcd61b1805f00cb50fb79f095fbcfdc6dc4e0bb5838579d6f7f1b482b WatchSource:0}: Error finding container bc0e24dbcd61b1805f00cb50fb79f095fbcfdc6dc4e0bb5838579d6f7f1b482b: Status 404 returned error can't find the container with id bc0e24dbcd61b1805f00cb50fb79f095fbcfdc6dc4e0bb5838579d6f7f1b482b
Sep 11 13:42:59.503481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.503449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:59.503561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.503536    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:59.503599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.503565    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:42:59.849698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.849661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerStarted","Data":"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"}
Sep 11 13:42:59.850174 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.849728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerStarted","Data":"bc0e24dbcd61b1805f00cb50fb79f095fbcfdc6dc4e0bb5838579d6f7f1b482b"}
Sep 11 13:42:59.885206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.885154    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podStartSLOduration=1.88513811 podStartE2EDuration="1.88513811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:59.88511647 +0000 UTC m=+1304.278803871" watchObservedRunningTime="2026-09-11 13:42:59.88513811 +0000 UTC m=+1304.278825520"
Sep 11 13:42:59.980839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:42:59.980810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:00.187446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.187355    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:00.192680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.192653    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:00.375514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.375461    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:43:00.375685 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.375531    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:43:00.854973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.854938    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:00.856118 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:00.856094    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:01.375397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:01.375367    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:43:01.375605 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:01.375421    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:43:01.591182 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:01.591145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:02.100708 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:02.100677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:02.195078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:02.195049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:02.375862 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:02.375779    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:43:02.375862 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:02.375843    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:43:03.228234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:03.228198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:03.375108 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:03.375071    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Sep 11 13:43:03.375300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:03.375127    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Sep 11 13:43:03.639978 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:03.639942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:43:04.381121 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:04.381080    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:43:04.382136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:04.382118    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:43:04.586919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:04.586893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:05.917699 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:05.917665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:06.914310 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:06.914280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:07.913360 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:07.913327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:07.987965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:07.987930    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:43:07.988154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:07.988132    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" podUID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" containerName="payload-processing" containerID="cri-o://155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe" gracePeriod=30
Sep 11 13:43:08.008347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.008321    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:43:08.008565 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.008543    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" podUID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" containerName="payload-pre-processing" containerID="cri-o://8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508" gracePeriod=30
Sep 11 13:43:08.059441 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.059410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.147065 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.147034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.258033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.257955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.278036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.278012    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:43:08.304857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.304817    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"]
Sep 11 13:43:08.309786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.309758    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-b65337-fm8mc"]
Sep 11 13:43:08.328013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.327987    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:43:08.365553 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.365520    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.421362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421279    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvxrn\" (UniqueName: \"kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn\") pod \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") "
Sep 11 13:43:08.421362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421344    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume\") pod \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\" (UID: \"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9\") "
Sep 11 13:43:08.421637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421382    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc9rv\" (UniqueName: \"kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv\") pod \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") "
Sep 11 13:43:08.421637 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421423    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume\") pod \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\" (UID: \"e6b7c0e2-0d13-44c8-89b6-2d491078d451\") "
Sep 11 13:43:08.421818 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421785    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6b7c0e2-0d13-44c8-89b6-2d491078d451" (UID: "e6b7c0e2-0d13-44c8-89b6-2d491078d451"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:08.421889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.421820    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" (UID: "e52023f0-47cb-4a69-98bd-eb8c89c6f5c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:08.428872 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.426593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv" (OuterVolumeSpecName: "kube-api-access-bc9rv") pod "e6b7c0e2-0d13-44c8-89b6-2d491078d451" (UID: "e6b7c0e2-0d13-44c8-89b6-2d491078d451"). InnerVolumeSpecName "kube-api-access-bc9rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:08.429184 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.429141    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn" (OuterVolumeSpecName: "kube-api-access-cvxrn") pod "e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" (UID: "e52023f0-47cb-4a69-98bd-eb8c89c6f5c9"). InnerVolumeSpecName "kube-api-access-cvxrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:08.450434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.450402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.522898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.522867    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:08.522898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.522894    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc9rv\" (UniqueName: \"kubernetes.io/projected/e6b7c0e2-0d13-44c8-89b6-2d491078d451-kube-api-access-bc9rv\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:08.522898 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.522903    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6b7c0e2-0d13-44c8-89b6-2d491078d451-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:08.523144 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.522911    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvxrn\" (UniqueName: \"kubernetes.io/projected/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9-kube-api-access-cvxrn\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:08.643134 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.643098    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:08.899772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.899736    2576 generic.go:356] "Generic (PLEG): container finished" podID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" containerID="155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe" exitCode=0
Sep 11 13:43:08.900007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.899779    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"
Sep 11 13:43:08.900007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.899814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" event={"ID":"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9","Type":"ContainerDied","Data":"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"}
Sep 11 13:43:08.900007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.899862    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q" event={"ID":"e52023f0-47cb-4a69-98bd-eb8c89c6f5c9","Type":"ContainerDied","Data":"6cc4bc3075ac55311e64f86e64af26eecf75d8405b8f403538cb55e50ae24d63"}
Sep 11 13:43:08.900007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.899887    2576 scope.go:117] "RemoveContainer" containerID="155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"
Sep 11 13:43:08.901212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.901189    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" containerID="8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508" exitCode=0
Sep 11 13:43:08.901321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.901230    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"
Sep 11 13:43:08.901321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.901257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" event={"ID":"e6b7c0e2-0d13-44c8-89b6-2d491078d451","Type":"ContainerDied","Data":"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"}
Sep 11 13:43:08.901321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.901293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn" event={"ID":"e6b7c0e2-0d13-44c8-89b6-2d491078d451","Type":"ContainerDied","Data":"742593dbd8663e8cdb988ef4c76a2bf4683e0b59b8d112ddfc45d2d8d0e52866"}
Sep 11 13:43:08.909301 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.909280    2576 scope.go:117] "RemoveContainer" containerID="155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"
Sep 11 13:43:08.909658 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:43:08.909639    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe\": container with ID starting with 155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe not found: ID does not exist" containerID="155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"
Sep 11 13:43:08.909759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.909672    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe"} err="failed to get container status \"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe\": rpc error: code = NotFound desc = could not find container \"155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe\": container with ID starting with 155bb909d6dc10eeba215c7294acd47d169255f12f543404fd136eb158a1c0fe not found: ID does not exist"
Sep 11 13:43:08.909759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.909689    2576 scope.go:117] "RemoveContainer" containerID="8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"
Sep 11 13:43:08.917775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.917754    2576 scope.go:117] "RemoveContainer" containerID="8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"
Sep 11 13:43:08.918043 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:43:08.918011    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508\": container with ID starting with 8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508 not found: ID does not exist" containerID="8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"
Sep 11 13:43:08.918043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.918031    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508"} err="failed to get container status \"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508\": rpc error: code = NotFound desc = could not find container \"8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508\": container with ID starting with 8f84b048d9babb776669fbb68da237e4e7d8b03006924ef7529831a527baa508 not found: ID does not exist"
Sep 11 13:43:08.923113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.923091    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:43:08.927490 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.927463    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-b65337-565947b7b5-hd69q"]
Sep 11 13:43:08.938733 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.938709    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:43:08.944147 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:08.944126    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-b65337-599c58b7db-nh8rn"]
Sep 11 13:43:09.601938 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:09.601903    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:09.703486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:09.703438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:10.121800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.121759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:10.247729 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.247697    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6f915b-d1f9-4435-b283-5fdb39e23432" path="/var/lib/kubelet/pods/aa6f915b-d1f9-4435-b283-5fdb39e23432/volumes"
Sep 11 13:43:10.248061 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.248041    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" path="/var/lib/kubelet/pods/e52023f0-47cb-4a69-98bd-eb8c89c6f5c9/volumes"
Sep 11 13:43:10.248356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.248345    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" path="/var/lib/kubelet/pods/e6b7c0e2-0d13-44c8-89b6-2d491078d451/volumes"
Sep 11 13:43:10.366281 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.366251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:10.484095 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.484001    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:10.730390 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730354    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:43:10.730753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730741    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" containerName="payload-processing"
Sep 11 13:43:10.730810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730755    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" containerName="payload-processing"
Sep 11 13:43:10.730810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730766    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" containerName="payload-pre-processing"
Sep 11 13:43:10.730810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730772    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" containerName="payload-pre-processing"
Sep 11 13:43:10.730903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730830    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6b7c0e2-0d13-44c8-89b6-2d491078d451" containerName="payload-pre-processing"
Sep 11 13:43:10.730903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.730837    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e52023f0-47cb-4a69-98bd-eb8c89c6f5c9" containerName="payload-processing"
Sep 11 13:43:10.733544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.733526    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.736189 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.736130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-c0f589fb-dockercfg-5tmpw\""
Sep 11 13:43:10.736379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.736363    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-c0f589fb\""
Sep 11 13:43:10.747692 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.747670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:43:10.844392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.844338    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bszrp\" (UniqueName: \"kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.844619 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.844406    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.945008 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.944974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bszrp\" (UniqueName: \"kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.945008 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.945013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.945775 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.945757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:10.955624 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:10.955593    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bszrp\" (UniqueName: \"kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp\") pod \"payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") " pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:11.043628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.043597    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:11.322185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.322156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:43:11.324863 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:43:11.324832    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33ffca4_8f97_463e_8135_b3036dd68491.slice/crio-efd59eb3387d492c2ae4d41266e720593885c72a60619c6c71857f7bb01489b7 WatchSource:0}: Error finding container efd59eb3387d492c2ae4d41266e720593885c72a60619c6c71857f7bb01489b7: Status 404 returned error can't find the container with id efd59eb3387d492c2ae4d41266e720593885c72a60619c6c71857f7bb01489b7
Sep 11 13:43:11.393944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.393909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:11.918271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.918240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" event={"ID":"f33ffca4-8f97-463e-8135-b3036dd68491","Type":"ContainerStarted","Data":"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"}
Sep 11 13:43:11.918482 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.918289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" event={"ID":"f33ffca4-8f97-463e-8135-b3036dd68491","Type":"ContainerStarted","Data":"efd59eb3387d492c2ae4d41266e720593885c72a60619c6c71857f7bb01489b7"}
Sep 11 13:43:11.918482 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.918331    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:11.941217 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:11.941162    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" podStartSLOduration=1.94114676 podStartE2EDuration="1.94114676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:11.938841067 +0000 UTC m=+1316.332528477" watchObservedRunningTime="2026-09-11 13:43:11.94114676 +0000 UTC m=+1316.334834171"
Sep 11 13:43:12.451082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:12.451048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:13.540945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:13.540917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:13.846536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:13.846426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:14.620036 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:14.620005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:15.613034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:15.612996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:16.702597 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:16.702565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:17.891297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:17.891261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:18.042355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:18.042327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:19.782663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:19.782631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:20.894236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:20.894209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:21.724380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:21.724347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:21.829612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:21.829582    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:22.925681 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:22.925645    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:43:22.940840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:22.940812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:24.335111 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:24.335083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:24.859361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:24.859329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:25.195795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:25.195712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:26.320025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:26.319988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:27.410324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:27.410286    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:28.021465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:28.021431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:28.120666 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:28.120631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:40.064039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.063998    2576 generic.go:356] "Generic (PLEG): container finished" podID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerID="b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12" exitCode=0
Sep 11 13:43:40.064454 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.064070    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerDied","Data":"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"}
Sep 11 13:43:40.064519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.064455    2576 scope.go:117] "RemoveContainer" containerID="b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"
Sep 11 13:43:40.065561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.065535    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:43:40.065623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.065597    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:43:40.065668 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:40.065656    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:43:41.071240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:41.071206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerStarted","Data":"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"}
Sep 11 13:43:41.186559 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:41.186516    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:41.187640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:41.187612    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:41.187764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:41.187663    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:42.187118 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:42.187080    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:42.187535 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:42.187146    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:43.187034 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:43.186992    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:43.187243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:43.187056    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:43.766433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:43.766392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:44.187663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:44.187572    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:44.187663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:44.187632    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:45.186931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:45.186895    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:45.187105 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:45.186951    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:45.854948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:45.854906    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:46.186981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:46.186877    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Sep 11 13:43:46.187159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:46.186977    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Sep 11 13:43:47.191902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:47.191865    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:47.193081 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:47.193061    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:43:49.175132 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:49.175095    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:49.365365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:49.365328    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:43:49.365629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:49.365604    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy" containerID="cri-o://504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314" gracePeriod=30
Sep 11 13:43:49.385366 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:49.385337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:50.139635 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:50.139601    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:51.312416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:51.312385    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:52.464876 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:52.464832    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:53.582177 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:53.582147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:54.616346 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.616324    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:43:54.766181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766227    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kbth\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766253    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766274    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766340    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766398    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766431    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766456    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data\") pod \"59523553-7bcf-4f40-b6f1-84377f8d2d89\" (UID: \"59523553-7bcf-4f40-b6f1-84377f8d2d89\") "
Sep 11 13:43:54.766687 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766603    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:54.766896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:54.766896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:54.766896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766877    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.766896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766883    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data" (OuterVolumeSpecName: "istio-data") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:54.766896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766897    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.767124 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766911    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.767124 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.766988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:54.768351 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.768327    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token" (OuterVolumeSpecName: "istio-token") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:54.768601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.768578    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth" (OuterVolumeSpecName: "kube-api-access-7kbth") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "kube-api-access-7kbth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:54.768826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.768807    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:43:54.768867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.768829    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59523553-7bcf-4f40-b6f1-84377f8d2d89" (UID: "59523553-7bcf-4f40-b6f1-84377f8d2d89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:54.867528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867481    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.867528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867526    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59523553-7bcf-4f40-b6f1-84377f8d2d89-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.867671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867537    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.867671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867545    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.867671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867554    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kbth\" (UniqueName: \"kubernetes.io/projected/59523553-7bcf-4f40-b6f1-84377f8d2d89-kube-api-access-7kbth\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.867671 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.867562    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59523553-7bcf-4f40-b6f1-84377f8d2d89-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:43:54.869663 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:54.869634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:55.141272 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.141185    2576 generic.go:356] "Generic (PLEG): container finished" podID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerID="504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314" exitCode=0
Sep 11 13:43:55.141272 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.141240    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"
Sep 11 13:43:55.141272 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.141247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerDied","Data":"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"}
Sep 11 13:43:55.141488 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.141298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh" event={"ID":"59523553-7bcf-4f40-b6f1-84377f8d2d89","Type":"ContainerDied","Data":"1afbf201a56b7e62061bf9b514f1efbeb81d7a46f88437355dabaf2fc4e40d57"}
Sep 11 13:43:55.141488 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.141329    2576 scope.go:117] "RemoveContainer" containerID="504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"
Sep 11 13:43:55.150348 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.150331    2576 scope.go:117] "RemoveContainer" containerID="807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"
Sep 11 13:43:55.157869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.157851    2576 scope.go:117] "RemoveContainer" containerID="504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"
Sep 11 13:43:55.158148 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:43:55.158127    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314\": container with ID starting with 504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314 not found: ID does not exist" containerID="504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"
Sep 11 13:43:55.158195 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.158156    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314"} err="failed to get container status \"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314\": rpc error: code = NotFound desc = could not find container \"504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314\": container with ID starting with 504f3b60e31a84f64368547053d88ed5a59aec87a55ca2b84870b8468d38e314 not found: ID does not exist"
Sep 11 13:43:55.158195 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.158172    2576 scope.go:117] "RemoveContainer" containerID="807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"
Sep 11 13:43:55.158416 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:43:55.158397    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea\": container with ID starting with 807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea not found: ID does not exist" containerID="807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"
Sep 11 13:43:55.158468 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.158420    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea"} err="failed to get container status \"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea\": rpc error: code = NotFound desc = could not find container \"807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea\": container with ID starting with 807a0491cd30163174071c78e102abcea550328a14b3cf8d20250fa59cadc1ea not found: ID does not exist"
Sep 11 13:43:55.169426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.169405    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:43:55.173630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:55.173606    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-b65337-openshift-default-6765579894-9c4vh"]
Sep 11 13:43:56.154936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:56.154873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:56.249164 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:56.249127    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" path="/var/lib/kubelet/pods/59523553-7bcf-4f40-b6f1-84377f8d2d89/volumes"
Sep 11 13:43:57.133666 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:57.133632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:43:58.151718 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:43:58.151687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:00.691471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:00.691440    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:00.753908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:00.753872    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:44:00.754105 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:00.754081    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" podUID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" containerName="payload-processing" containerID="cri-o://ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816" gracePeriod=30
Sep 11 13:44:00.811157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:00.811119    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:00.884132 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:00.884103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:01.001012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.000990    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:44:01.012271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.012249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xclj4\" (UniqueName: \"kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4\") pod \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") "
Sep 11 13:44:01.012415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.012366    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume\") pod \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\" (UID: \"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f\") "
Sep 11 13:44:01.012720 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.012698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" (UID: "0738c803-8ef9-4bb2-ac3c-fc16dbefca0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:44:01.014432 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.014408    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4" (OuterVolumeSpecName: "kube-api-access-xclj4") pod "0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" (UID: "0738c803-8ef9-4bb2-ac3c-fc16dbefca0f"). InnerVolumeSpecName "kube-api-access-xclj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:44:01.087759 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.087717    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:01.113350 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.113300    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:01.113350 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.113338    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xclj4\" (UniqueName: \"kubernetes.io/projected/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f-kube-api-access-xclj4\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:01.173510 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.173462    2576 generic.go:356] "Generic (PLEG): container finished" podID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" containerID="ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816" exitCode=0
Sep 11 13:44:01.173649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.173528    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"
Sep 11 13:44:01.173649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.173537    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" event={"ID":"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f","Type":"ContainerDied","Data":"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"}
Sep 11 13:44:01.173649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.173584    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c" event={"ID":"0738c803-8ef9-4bb2-ac3c-fc16dbefca0f","Type":"ContainerDied","Data":"8ac64958a38399918c007ac55af69f13d8acc8148e97a8313371b781a443561e"}
Sep 11 13:44:01.173649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.173610    2576 scope.go:117] "RemoveContainer" containerID="ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"
Sep 11 13:44:01.182172 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.182155    2576 scope.go:117] "RemoveContainer" containerID="ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"
Sep 11 13:44:01.182465 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:44:01.182445    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816\": container with ID starting with ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816 not found: ID does not exist" containerID="ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"
Sep 11 13:44:01.182538 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.182473    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816"} err="failed to get container status \"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816\": rpc error: code = NotFound desc = could not find container \"ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816\": container with ID starting with ceda65d313bccdb9f96f669b0720909d89e1754065fb5aade62b2502bb89a816 not found: ID does not exist"
Sep 11 13:44:01.194690 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.194664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:01.209737 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.209712    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:44:01.215782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.215761    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a649d3-68f5cc6cf7-65f8c"]
Sep 11 13:44:01.627923 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.627895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:01.911347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:01.911256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:02.248444 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:02.248364    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" path="/var/lib/kubelet/pods/0738c803-8ef9-4bb2-ac3c-fc16dbefca0f/volumes"
Sep 11 13:44:03.366022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:03.365988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:04.678248 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:04.678219    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:05.618255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:05.618217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:06.508834 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:06.508801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:07.808568 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:07.808541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:08.817547 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:08.817514    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:08.891282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:08.891247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:09.165947 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:09.165860    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:09.175193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:09.175166    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:44:09.175382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:09.175360    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" containerID="cri-o://31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf" gracePeriod=30
Sep 11 13:44:09.747903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:09.747870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:10.175448 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175412    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175770    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" containerName="payload-processing"
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175785    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" containerName="payload-processing"
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175796    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175802    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175809    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.175857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175814    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.176071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175871    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.176071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175881    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="59523553-7bcf-4f40-b6f1-84377f8d2d89" containerName="istio-proxy"
Sep 11 13:44:10.176071 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.175889    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0738c803-8ef9-4bb2-ac3c-fc16dbefca0f" containerName="payload-processing"
Sep 11 13:44:10.180048 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.180030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.185398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.185369    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-0317c3-openshift-default-dockercfg-zdgtm\""
Sep 11 13:44:10.210985 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.210947    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:44:10.242804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.242759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:10.283845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.283792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.283854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.283893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.283931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.283983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.284003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.284043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msw75\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.284087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.284364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.284132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.373237 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.373203    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:10.384549 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384668 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384712 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384712 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msw75\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384796 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.384965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.385057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.384945    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.385057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.385168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385059    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.385233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385092    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.390972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.390972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.390972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.385799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.392787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.392758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.393277 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.393257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.398213 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.398192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.406641 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.406622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msw75\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75\") pod \"e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") " pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.490279 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.490210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:10.781358 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.781334    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:44:10.783066 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:44:10.783041    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b74d74_1b5e_45d5_9a3c_8b78900521b2.slice/crio-573de0aa9c05d179e9c024f826263ce1646f806c745abe2a57ed1d14b9149990 WatchSource:0}: Error finding container 573de0aa9c05d179e9c024f826263ce1646f806c745abe2a57ed1d14b9149990: Status 404 returned error can't find the container with id 573de0aa9c05d179e9c024f826263ce1646f806c745abe2a57ed1d14b9149990
Sep 11 13:44:10.785268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.785233    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:10.785343 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.785297    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:10.785343 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:10.785326    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:11.227761 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:11.227662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerStarted","Data":"7362ac0ef5efb001ea2c007f636436db28d3b027be0e0a8863f1622a20bc571c"}
Sep 11 13:44:11.227761 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:11.227714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerStarted","Data":"573de0aa9c05d179e9c024f826263ce1646f806c745abe2a57ed1d14b9149990"}
Sep 11 13:44:11.491409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:11.491333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:11.496420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:11.496397    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:11.540196 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:11.540145    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podStartSLOduration=1.540128607 podStartE2EDuration="1.540128607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:11.279563337 +0000 UTC m=+1375.673250747" watchObservedRunningTime="2026-09-11 13:44:11.540128607 +0000 UTC m=+1375.933816018"
Sep 11 13:44:12.232909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:12.232877    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:12.233913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:12.233892    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:12.263300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:12.263268    2576 patch_prober.go:28] interesting pod/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:44:12.263482 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:12.263316    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:44:13.048086 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:13.048052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:13.333173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:13.333077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:13.486428 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:13.486399    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:14.421634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.421612    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:44:14.519215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519193    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519223    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519287    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519303    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx52s\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519338    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519392    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519450    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token\") pod \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\" (UID: \"0a4f3b99-6e8d-440a-a067-e687bf7c21ab\") "
Sep 11 13:44:14.519727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519534    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:44:14.519727 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519643    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data" (OuterVolumeSpecName: "istio-data") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519764    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519652    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519874    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519896    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519912    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.519936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.519925    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.521697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.521510    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:44:14.521697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.521510    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:44:14.521697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.521595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s" (OuterVolumeSpecName: "kube-api-access-gx52s") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "kube-api-access-gx52s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:44:14.521870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.521787    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token" (OuterVolumeSpecName: "istio-token") pod "0a4f3b99-6e8d-440a-a067-e687bf7c21ab" (UID: "0a4f3b99-6e8d-440a-a067-e687bf7c21ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:44:14.620588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.620557    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.620588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.620582    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.620588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.620591    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx52s\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-kube-api-access-gx52s\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.620785 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.620601    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.620785 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.620610    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4f3b99-6e8d-440a-a067-e687bf7c21ab-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:44:14.906813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:14.906732    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:15.248064 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.247975    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerID="31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf" exitCode=0
Sep 11 13:44:15.248064 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.248010    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"
Sep 11 13:44:15.248246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.248063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerDied","Data":"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"}
Sep 11 13:44:15.248246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.248110    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr" event={"ID":"0a4f3b99-6e8d-440a-a067-e687bf7c21ab","Type":"ContainerDied","Data":"cf03eb01c6223ab60390866d49a3721c16d5be199e2e32377565236a3657ede2"}
Sep 11 13:44:15.248246 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.248132    2576 scope.go:117] "RemoveContainer" containerID="31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"
Sep 11 13:44:15.257119 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.257094    2576 scope.go:117] "RemoveContainer" containerID="3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"
Sep 11 13:44:15.264646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.264632    2576 scope.go:117] "RemoveContainer" containerID="31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"
Sep 11 13:44:15.264894 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:44:15.264876    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf\": container with ID starting with 31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf not found: ID does not exist" containerID="31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"
Sep 11 13:44:15.264961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.264901    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf"} err="failed to get container status \"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf\": rpc error: code = NotFound desc = could not find container \"31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf\": container with ID starting with 31c914193b09d7b67b465e48defa932827c2f166f78c9f67448276a9c44570bf not found: ID does not exist"
Sep 11 13:44:15.264961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.264920    2576 scope.go:117] "RemoveContainer" containerID="3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"
Sep 11 13:44:15.265172 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:44:15.265156    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40\": container with ID starting with 3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40 not found: ID does not exist" containerID="3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"
Sep 11 13:44:15.265214 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.265179    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40"} err="failed to get container status \"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40\": rpc error: code = NotFound desc = could not find container \"3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40\": container with ID starting with 3b883319a03bf541442c7642d790a5ea56d4efce112b5074fe72df14a1601b40 not found: ID does not exist"
Sep 11 13:44:15.287555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.287534    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:44:15.298638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:15.298613    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-a649d3-openshift-default-c66b99466-p2tlr"]
Sep 11 13:44:16.207287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:16.207251    2576 scope.go:117] "RemoveContainer" containerID="cb56410166171726feb37006e3a03a0fd17a433abb68acf657a4d3ddf9ff7986"
Sep 11 13:44:16.247925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:16.247901    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" path="/var/lib/kubelet/pods/0a4f3b99-6e8d-440a-a067-e687bf7c21ab/volumes"
Sep 11 13:44:16.328369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:16.328341    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:17.400860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:17.400831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:18.308662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:18.308628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:19.392437 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:19.392407    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:20.318433 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:20.318400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:20.405692 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:20.405662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:20.679117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:20.679034    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:20.989813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:20.989739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:21.089639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.089611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:21.206717 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.206684    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:44:21.207035 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207024    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.207080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207038    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.207080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207048    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.207080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207054    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.207179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207105    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.207179 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.207113    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4f3b99-6e8d-440a-a067-e687bf7c21ab" containerName="istio-proxy"
Sep 11 13:44:21.211349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.211332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.214137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.214117    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-0317c3-dockercfg-l7ts6\""
Sep 11 13:44:21.214457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.214443    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-0317c3\""
Sep 11 13:44:21.227352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.227332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:44:21.280803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.280763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7pq2\" (UniqueName: \"kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.281615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.280852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.382157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.382127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7pq2\" (UniqueName: \"kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.382324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.382178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.382841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.382822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.395990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.395970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7pq2\" (UniqueName: \"kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2\") pod \"payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.521800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.521773    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:21.796422 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:21.796353    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:44:21.799665 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:44:21.799631    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052bebd0_f531_4be3_9548_f2f7ffe007f8.slice/crio-9279612c935b30872e2370cb7142eca84683d7f65343ed8fc3edcdd742bdf214 WatchSource:0}: Error finding container 9279612c935b30872e2370cb7142eca84683d7f65343ed8fc3edcdd742bdf214: Status 404 returned error can't find the container with id 9279612c935b30872e2370cb7142eca84683d7f65343ed8fc3edcdd742bdf214
Sep 11 13:44:22.284807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:22.284773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" event={"ID":"052bebd0-f531-4be3-9548-f2f7ffe007f8","Type":"ContainerStarted","Data":"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"}
Sep 11 13:44:22.284981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:22.284832    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" event={"ID":"052bebd0-f531-4be3-9548-f2f7ffe007f8","Type":"ContainerStarted","Data":"9279612c935b30872e2370cb7142eca84683d7f65343ed8fc3edcdd742bdf214"}
Sep 11 13:44:22.284981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:22.284871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:22.306581 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:22.306523    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" podStartSLOduration=1.3064924549999999 podStartE2EDuration="1.306492455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:22.305741026 +0000 UTC m=+1386.699428454" watchObservedRunningTime="2026-09-11 13:44:22.306492455 +0000 UTC m=+1386.700179875"
Sep 11 13:44:24.444483 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:24.444449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:33.291680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:33.291646    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:44:43.274707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:43.274654    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 13:44:43.275135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:43.274732    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 13:44:43.396531 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:43.396476    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:43.505407 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:43.505375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:43.770368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:43.770333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:44.124200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.124120    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 13:44:44.128680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.128663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.132091 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.132072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-9edb53-openshift-default-dockercfg-qghc8\""
Sep 11 13:44:44.145371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.145348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 13:44:44.172678 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.172647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:44.297841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.297801    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.297853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.297884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.297912    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.298015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.298054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtl59\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.298096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.298189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.298275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.298222    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.341456 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.341410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:44.399906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.399788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.399906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.399866    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.399906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.399894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.399919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.399990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtl59\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400863 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400832    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.400966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.400904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.401260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.401203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.401356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.401263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.401528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.401485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.404172 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.404118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.406526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.406432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.412155 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.411793    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.414420 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.414388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtl59\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59\") pod \"e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") " pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.439243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.439209    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:44.563970 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.563918    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:44.719998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.719973    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 13:44:44.722639 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:44:44.722599    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e24c047_5ccf_4f37_8cb3_9963f173544d.slice/crio-70025309ecb83856af5fb5feb35f341e5654af7c050202bf40f56a2fe98ae0c8 WatchSource:0}: Error finding container 70025309ecb83856af5fb5feb35f341e5654af7c050202bf40f56a2fe98ae0c8: Status 404 returned error can't find the container with id 70025309ecb83856af5fb5feb35f341e5654af7c050202bf40f56a2fe98ae0c8
Sep 11 13:44:44.725355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.725311    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:44.725441 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.725398    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:44.725480 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:44.725440    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:45.298379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.298343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:45.396988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.396956    2576 generic.go:356] "Generic (PLEG): container finished" podID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerID="7362ac0ef5efb001ea2c007f636436db28d3b027be0e0a8863f1622a20bc571c" exitCode=0
Sep 11 13:44:45.397158 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.397031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerDied","Data":"7362ac0ef5efb001ea2c007f636436db28d3b027be0e0a8863f1622a20bc571c"}
Sep 11 13:44:45.397544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.397526    2576 scope.go:117] "RemoveContainer" containerID="7362ac0ef5efb001ea2c007f636436db28d3b027be0e0a8863f1622a20bc571c"
Sep 11 13:44:45.398713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.398689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerStarted","Data":"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"}
Sep 11 13:44:45.398806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.398744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerStarted","Data":"70025309ecb83856af5fb5feb35f341e5654af7c050202bf40f56a2fe98ae0c8"}
Sep 11 13:44:45.398848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.398810    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:45.398886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.398875    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:45.398924 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.398907    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:44:45.439505 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.439431    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:45.444548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.444525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:45.453146 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:45.453099    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podStartSLOduration=1.45308029 podStartE2EDuration="1.45308029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:45.452489897 +0000 UTC m=+1409.846177309" watchObservedRunningTime="2026-09-11 13:44:45.45308029 +0000 UTC m=+1409.846767704"
Sep 11 13:44:46.299616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.299590    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:46.405192 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.405160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerStarted","Data":"6d9ac25b9761f9651bba1a7e930abf59d5054fc442adbd7ef75ae1caad4805a7"}
Sep 11 13:44:46.405509 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.405478    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:46.406235 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.406216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:44:46.490377 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.490346    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:46.490590 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.490568    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:46.490652 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:46.490620    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:47.439957 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:47.439925    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:47.491037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:47.491004    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:47.491207 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:47.491060    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:47.565421 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:47.565384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:47.709035 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:47.708941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:48.490973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:48.490941    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:48.491360 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:48.490995    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:49.491582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:49.491547    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:49.491971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:49.491599    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:49.562747 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:49.562715    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:50.491452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:50.491417    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:50.491649 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:50.491470    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:50.639638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:50.639602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:51.491465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:51.491431    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Sep 11 13:44:51.491676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:51.491492    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Sep 11 13:44:51.811981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:51.811943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:51.907025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:51.906986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:52.188376 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.188291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:52.513379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.513256    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:52.514409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.514352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:52.517548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.517517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:44:52.553388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.553364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:52.653859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.653828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:52.747996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.747964    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:44:52.750357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.750342    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.752786 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.752765    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-9edb53\""
Sep 11 13:44:52.752904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.752823    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-9edb53-dockercfg-lcpqg\""
Sep 11 13:44:52.764509 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.764427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:44:52.778916 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.778893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg8zw\" (UniqueName: \"kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.779051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.778949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.879920 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.879880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg8zw\" (UniqueName: \"kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.880114 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.879995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.880890 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.880864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:52.891161 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:52.891138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg8zw\" (UniqueName: \"kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw\") pod \"payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:53.060431 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.060393    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:53.309328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.309296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:53.334184 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.334165    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:44:53.336898 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:44:53.336871    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52034605_5fcd_4899_aa52_4654431bb09e.slice/crio-ead2d807df3ccc9b38566a05a91fc16174dcf8ae13867eb20d45c7fb7c6faeb4 WatchSource:0}: Error finding container ead2d807df3ccc9b38566a05a91fc16174dcf8ae13867eb20d45c7fb7c6faeb4: Status 404 returned error can't find the container with id ead2d807df3ccc9b38566a05a91fc16174dcf8ae13867eb20d45c7fb7c6faeb4
Sep 11 13:44:53.456852 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.456820    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" event={"ID":"52034605-5fcd-4899-aa52-4654431bb09e","Type":"ContainerStarted","Data":"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"}
Sep 11 13:44:53.456995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.456873    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:44:53.456995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.456890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" event={"ID":"52034605-5fcd-4899-aa52-4654431bb09e","Type":"ContainerStarted","Data":"ead2d807df3ccc9b38566a05a91fc16174dcf8ae13867eb20d45c7fb7c6faeb4"}
Sep 11 13:44:53.482536 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:53.482474    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" podStartSLOduration=1.482459828 podStartE2EDuration="1.482459828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:53.480012825 +0000 UTC m=+1417.873700236" watchObservedRunningTime="2026-09-11 13:44:53.482459828 +0000 UTC m=+1417.876147238"
Sep 11 13:44:54.198971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:54.198936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:55.309971 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:55.309943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:56.348554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:56.348515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:44:57.242373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:44:57.242343    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:00.195584 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.195550    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"]
Sep 11 13:45:00.198289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.198270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"
Sep 11 13:45:00.202605 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.202585    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:45:00.207701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.207682    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"
Sep 11 13:45:00.213368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.213346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"]
Sep 11 13:45:00.233303 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.233267    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"]
Sep 11 13:45:00.237091 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.237065    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"]
Sep 11 13:45:00.237226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.237193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"
Sep 11 13:45:00.240255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.240232    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"
Sep 11 13:45:00.255789 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.255761    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"
Sep 11 13:45:00.268264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.268233    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"
Sep 11 13:45:00.278078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.275831    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"]
Sep 11 13:45:00.278878 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.278850    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"]
Sep 11 13:45:00.293183 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.291634    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"]
Sep 11 13:45:00.305892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.303477    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"]
Sep 11 13:45:00.305892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.303626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"
Sep 11 13:45:00.322174 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.321726    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"
Sep 11 13:45:00.490491 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.490469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"]
Sep 11 13:45:00.492479 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:45:00.492455    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode536965e_bf3e_4f23_8429_09a71d878b3c.slice/crio-04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e WatchSource:0}: Error finding container 04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e: Status 404 returned error can't find the container with id 04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e
Sep 11 13:45:00.552980 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.552956    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"]
Sep 11 13:45:00.554769 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:45:00.554746    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa317d58_6e2e_40ef_b68e_37d76cb0749b.slice/crio-b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a WatchSource:0}: Error finding container b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a: Status 404 returned error can't find the container with id b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a
Sep 11 13:45:00.574486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.574465    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"]
Sep 11 13:45:00.588465 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:45:00.588443    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47223586_fba3_4ff3_97a7_20cac4d537d3.slice/crio-1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d WatchSource:0}: Error finding container 1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d: Status 404 returned error can't find the container with id 1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d
Sep 11 13:45:00.625299 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:00.625267    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"]
Sep 11 13:45:00.627199 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:45:00.627175    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eeb6c2a_f8a5_49a4_9c53_141968bccc32.slice/crio-966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b WatchSource:0}: Error finding container 966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b: Status 404 returned error can't find the container with id 966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b
Sep 11 13:45:01.497966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.497927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6" event={"ID":"47223586-fba3-4ff3-97a7-20cac4d537d3","Type":"ContainerStarted","Data":"99e818c85d3c0228e55f4dff32506eac95c7b7be4da990a28152453d8cb745fd"}
Sep 11 13:45:01.498386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.497983    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6" event={"ID":"47223586-fba3-4ff3-97a7-20cac4d537d3","Type":"ContainerStarted","Data":"1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d"}
Sep 11 13:45:01.499349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.499324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n" event={"ID":"e536965e-bf3e-4f23-8429-09a71d878b3c","Type":"ContainerStarted","Data":"e47796093bee3eef933fc7977401c6b00cf8dd5488ae7ef61dcd098e6fb53398"}
Sep 11 13:45:01.499418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.499368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n" event={"ID":"e536965e-bf3e-4f23-8429-09a71d878b3c","Type":"ContainerStarted","Data":"04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e"}
Sep 11 13:45:01.500691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.500669    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg" event={"ID":"1eeb6c2a-f8a5-49a4-9c53-141968bccc32","Type":"ContainerStarted","Data":"3a5a6ce67b6e583ea9c0538fcf20a7434de5ccff8354909af6a17b016028d3c4"}
Sep 11 13:45:01.500808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.500710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg" event={"ID":"1eeb6c2a-f8a5-49a4-9c53-141968bccc32","Type":"ContainerStarted","Data":"966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b"}
Sep 11 13:45:01.502018 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.501995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj" event={"ID":"aa317d58-6e2e-40ef-b68e-37d76cb0749b","Type":"ContainerStarted","Data":"e831a2625f90372fcdb52e3e3e07bee7e3d175f1aa100c151a2e0b7cb5b1e8a2"}
Sep 11 13:45:01.502109 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.502039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj" event={"ID":"aa317d58-6e2e-40ef-b68e-37d76cb0749b","Type":"ContainerStarted","Data":"b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a"}
Sep 11 13:45:01.525842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.525799    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6" podStartSLOduration=1.5257853460000002 podStartE2EDuration="1.525785346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.525335039 +0000 UTC m=+1425.919022451" watchObservedRunningTime="2026-09-11 13:45:01.525785346 +0000 UTC m=+1425.919472756"
Sep 11 13:45:01.594834 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.594784    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg" podStartSLOduration=1.594770655 podStartE2EDuration="1.594770655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.566377744 +0000 UTC m=+1425.960065166" watchObservedRunningTime="2026-09-11 13:45:01.594770655 +0000 UTC m=+1425.988458065"
Sep 11 13:45:01.595040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.595020    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n" podStartSLOduration=1.595016955 podStartE2EDuration="1.595016955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.593386569 +0000 UTC m=+1425.987073981" watchObservedRunningTime="2026-09-11 13:45:01.595016955 +0000 UTC m=+1425.988704404"
Sep 11 13:45:01.626063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:01.626018    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj" podStartSLOduration=1.626003756 podStartE2EDuration="1.626003756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:45:01.625151222 +0000 UTC m=+1426.018838633" watchObservedRunningTime="2026-09-11 13:45:01.626003756 +0000 UTC m=+1426.019691167"
Sep 11 13:45:02.508460 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.508422    2576 generic.go:356] "Generic (PLEG): container finished" podID="47223586-fba3-4ff3-97a7-20cac4d537d3" containerID="99e818c85d3c0228e55f4dff32506eac95c7b7be4da990a28152453d8cb745fd" exitCode=0
Sep 11 13:45:02.508914 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.508521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6" event={"ID":"47223586-fba3-4ff3-97a7-20cac4d537d3","Type":"ContainerDied","Data":"99e818c85d3c0228e55f4dff32506eac95c7b7be4da990a28152453d8cb745fd"}
Sep 11 13:45:02.509973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.509953    2576 generic.go:356] "Generic (PLEG): container finished" podID="e536965e-bf3e-4f23-8429-09a71d878b3c" containerID="e47796093bee3eef933fc7977401c6b00cf8dd5488ae7ef61dcd098e6fb53398" exitCode=0
Sep 11 13:45:02.510040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.510029    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n" event={"ID":"e536965e-bf3e-4f23-8429-09a71d878b3c","Type":"ContainerDied","Data":"e47796093bee3eef933fc7977401c6b00cf8dd5488ae7ef61dcd098e6fb53398"}
Sep 11 13:45:02.511394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.511377    2576 generic.go:356] "Generic (PLEG): container finished" podID="1eeb6c2a-f8a5-49a4-9c53-141968bccc32" containerID="3a5a6ce67b6e583ea9c0538fcf20a7434de5ccff8354909af6a17b016028d3c4" exitCode=0
Sep 11 13:45:02.511445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.511402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg" event={"ID":"1eeb6c2a-f8a5-49a4-9c53-141968bccc32","Type":"ContainerDied","Data":"3a5a6ce67b6e583ea9c0538fcf20a7434de5ccff8354909af6a17b016028d3c4"}
Sep 11 13:45:02.512826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.512803    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa317d58-6e2e-40ef-b68e-37d76cb0749b" containerID="e831a2625f90372fcdb52e3e3e07bee7e3d175f1aa100c151a2e0b7cb5b1e8a2" exitCode=0
Sep 11 13:45:02.512933 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:02.512833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj" event={"ID":"aa317d58-6e2e-40ef-b68e-37d76cb0749b","Type":"ContainerDied","Data":"e831a2625f90372fcdb52e3e3e07bee7e3d175f1aa100c151a2e0b7cb5b1e8a2"}
Sep 11 13:45:03.662516 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:03.662478    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"
Sep 11 13:45:03.719995 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:03.719966    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"
Sep 11 13:45:03.743095 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:03.743073    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"
Sep 11 13:45:03.746253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:03.746224    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"
Sep 11 13:45:04.463057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.463029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:45:04.529701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.529129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"
Sep 11 13:45:04.529701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.529616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6" event={"ID":"47223586-fba3-4ff3-97a7-20cac4d537d3","Type":"ContainerDied","Data":"1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d"}
Sep 11 13:45:04.529701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.529671    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d7e9488454e54100e5dff02eed2ce64cec2dbe2fcb1212d86e3c46f907cdc8d"
Sep 11 13:45:04.532117 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.532091    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"
Sep 11 13:45:04.532255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.532130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n" event={"ID":"e536965e-bf3e-4f23-8429-09a71d878b3c","Type":"ContainerDied","Data":"04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e"}
Sep 11 13:45:04.532255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.532173    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04b55a8f1609a557c7c10dcd1964073920e33346a4fc18ff9cbdb5a9c652731e"
Sep 11 13:45:04.533679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.533660    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"
Sep 11 13:45:04.533770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.533687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg" event={"ID":"1eeb6c2a-f8a5-49a4-9c53-141968bccc32","Type":"ContainerDied","Data":"966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b"}
Sep 11 13:45:04.533770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.533725    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="966c1616e3544236270330f4f46087a6fa7296747932e55ea7f17429e6fb514b"
Sep 11 13:45:04.535227 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.535212    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"
Sep 11 13:45:04.535313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.535249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj" event={"ID":"aa317d58-6e2e-40ef-b68e-37d76cb0749b","Type":"ContainerDied","Data":"b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a"}
Sep 11 13:45:04.535313 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:04.535283    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2ffcaf186a545796c35a413cee99b64f5c18654fc3bf386d4607265098bc81a"
Sep 11 13:45:05.053643 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:05.053609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:13.588232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.588199    2576 generic.go:356] "Generic (PLEG): container finished" podID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerID="0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1" exitCode=0
Sep 11 13:45:13.588659 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.588272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerDied","Data":"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"}
Sep 11 13:45:13.588709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.588661    2576 scope.go:117] "RemoveContainer" containerID="0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"
Sep 11 13:45:13.589869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.589840    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:45:13.589941 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.589916    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:45:13.589977 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:13.589946    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:45:14.594824 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:14.594790    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerStarted","Data":"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"}
Sep 11 13:45:15.224110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:15.224052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:15.439544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:15.439483    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:45:15.440713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:15.440689    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:45:15.440766 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:15.440739    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 13:45:16.236282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:16.236252    2576 scope.go:117] "RemoveContainer" containerID="5746afb8971eb0f3c6c1ee8155b47db9bce9cee8620b58115109faa3094e054e"
Sep 11 13:45:16.255673 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:16.255649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:16.413612 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:16.413579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:45:16.440215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:16.440183    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:45:16.440354 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:16.440234    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 13:45:17.242512 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:17.242474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:17.440244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:17.440203    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:45:17.440439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:17.440277    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 13:45:17.497223 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:17.497138    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:18.440046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:18.440013    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:45:18.440412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:18.440068    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 13:45:18.674808 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:18.674776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:19.439555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:19.439522    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:45:19.439736 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:19.439576    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Sep 11 13:45:19.823970 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:19.823940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:20.444512 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:20.444479    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:45:20.445787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:20.445762    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:45:20.876656 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:20.876605    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:22.046416 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:22.046383    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:23.133345 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:23.133312    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:26.641156 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:26.641126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:48.255357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:48.255327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:45:49.921840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:45:49.921804    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:00.736642 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.736608    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"]
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.736962    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa317d58-6e2e-40ef-b68e-37d76cb0749b" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.736973    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa317d58-6e2e-40ef-b68e-37d76cb0749b" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.736989    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eeb6c2a-f8a5-49a4-9c53-141968bccc32" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.736995    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eeb6c2a-f8a5-49a4-9c53-141968bccc32" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737002    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e536965e-bf3e-4f23-8429-09a71d878b3c" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737007    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e536965e-bf3e-4f23-8429-09a71d878b3c" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737018    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47223586-fba3-4ff3-97a7-20cac4d537d3" containerName="cleanup"
Sep 11 13:46:00.737057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737023    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47223586-fba3-4ff3-97a7-20cac4d537d3" containerName="cleanup"
Sep 11 13:46:00.737309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737085    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa317d58-6e2e-40ef-b68e-37d76cb0749b" containerName="cleanup"
Sep 11 13:46:00.737309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737093    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eeb6c2a-f8a5-49a4-9c53-141968bccc32" containerName="cleanup"
Sep 11 13:46:00.737309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737100    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e536965e-bf3e-4f23-8429-09a71d878b3c" containerName="cleanup"
Sep 11 13:46:00.737309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.737106    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="47223586-fba3-4ff3-97a7-20cac4d537d3" containerName="cleanup"
Sep 11 13:46:00.740002 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.739987    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:00.742750 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.742729    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:46:00.743265 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.743251    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:46:00.757134 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.757111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"]
Sep 11 13:46:00.918428 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:00.918390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78\" (UID: \"55573de4-6435-402a-8de0-b90748a3d7dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:01.019521 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.019464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78\" (UID: \"55573de4-6435-402a-8de0-b90748a3d7dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:01.020060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.020040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78\" (UID: \"55573de4-6435-402a-8de0-b90748a3d7dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:01.049478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.049456    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:01.066714 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.066690    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:01.325677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.325653    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"]
Sep 11 13:46:01.327535 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:46:01.327504    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55573de4_6435_402a_8de0_b90748a3d7dd.slice/crio-66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b WatchSource:0}: Error finding container 66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b: Status 404 returned error can't find the container with id 66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b
Sep 11 13:46:01.838334 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.838295    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78" event={"ID":"55573de4-6435-402a-8de0-b90748a3d7dd","Type":"ContainerStarted","Data":"ffb87df00a569f409d4bac337c1338d847a8a5a639a6ef216f8ec04af180be3f"}
Sep 11 13:46:01.838743 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:01.838357    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78" event={"ID":"55573de4-6435-402a-8de0-b90748a3d7dd","Type":"ContainerStarted","Data":"66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b"}
Sep 11 13:46:02.137591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:02.137555    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:02.844530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:02.844481    2576 generic.go:356] "Generic (PLEG): container finished" podID="55573de4-6435-402a-8de0-b90748a3d7dd" containerID="ffb87df00a569f409d4bac337c1338d847a8a5a639a6ef216f8ec04af180be3f" exitCode=0
Sep 11 13:46:02.844901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:02.844559    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78" event={"ID":"55573de4-6435-402a-8de0-b90748a3d7dd","Type":"ContainerDied","Data":"ffb87df00a569f409d4bac337c1338d847a8a5a639a6ef216f8ec04af180be3f"}
Sep 11 13:46:03.745782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:03.745748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:03.975791 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:03.975767    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:04.151243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.151205    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca\") pod \"55573de4-6435-402a-8de0-b90748a3d7dd\" (UID: \"55573de4-6435-402a-8de0-b90748a3d7dd\") "
Sep 11 13:46:04.151575 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.151553    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55573de4-6435-402a-8de0-b90748a3d7dd" (UID: "55573de4-6435-402a-8de0-b90748a3d7dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:46:04.251844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.251809    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55573de4-6435-402a-8de0-b90748a3d7dd-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:46:04.855623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.855596    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"
Sep 11 13:46:04.855802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.855628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78" event={"ID":"55573de4-6435-402a-8de0-b90748a3d7dd","Type":"ContainerDied","Data":"66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b"}
Sep 11 13:46:04.855802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:04.855673    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66afd211e17b3bae5f6017871cf95cb3e3318413f553274d1205d5b796cce40b"
Sep 11 13:46:05.085167 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:05.085139    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:06.194399 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.194367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:06.278182 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.278153    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:46:06.278526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.278513    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55573de4-6435-402a-8de0-b90748a3d7dd" containerName="revoke-keys"
Sep 11 13:46:06.278582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.278527    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55573de4-6435-402a-8de0-b90748a3d7dd" containerName="revoke-keys"
Sep 11 13:46:06.278615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.278593    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55573de4-6435-402a-8de0-b90748a3d7dd" containerName="revoke-keys"
Sep 11 13:46:06.281530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.281490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.286935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.286912    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"auth-test-model-0317c3-kserve-self-signed-certs\""
Sep 11 13:46:06.287070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.286946    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"openshift-service-ca.crt\""
Sep 11 13:46:06.287070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.286946    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"default-dockercfg-8bq8v\""
Sep 11 13:46:06.287070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.287017    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-0317c3\"/\"kube-root-ca.crt\""
Sep 11 13:46:06.313658 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.313624    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:46:06.365935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.365909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.366090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.365959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.366090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.365989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.366090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.366024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.366090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.366048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt2zk\" (UniqueName: \"kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.366255 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.366094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.395419 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.395390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:06.467185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467126    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt2zk\" (UniqueName: \"kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467712    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.467840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.467776    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.469463 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.469438    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.469744 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.469730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.480631 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.480612    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt2zk\" (UniqueName: \"kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk\") pod \"auth-test-model-0317c3-kserve-6d7f774c99-tbdfp\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") " pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.590896 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.590861    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:06.873604 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:06.873577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:46:06.875942 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:46:06.875913    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod688e62fb_00f5_4839_983d_1a1b8f123d27.slice/crio-7c50c28d640b5173f3b25ff60c8be996e9b9cac743b0a7858fda17d876b30f07 WatchSource:0}: Error finding container 7c50c28d640b5173f3b25ff60c8be996e9b9cac743b0a7858fda17d876b30f07: Status 404 returned error can't find the container with id 7c50c28d640b5173f3b25ff60c8be996e9b9cac743b0a7858fda17d876b30f07
Sep 11 13:46:07.630982 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:07.630947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:07.875667 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:07.875629    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" event={"ID":"688e62fb-00f5-4839-983d-1a1b8f123d27","Type":"ContainerStarted","Data":"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"}
Sep 11 13:46:07.875667 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:07.875674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" event={"ID":"688e62fb-00f5-4839-983d-1a1b8f123d27","Type":"ContainerStarted","Data":"7c50c28d640b5173f3b25ff60c8be996e9b9cac743b0a7858fda17d876b30f07"}
Sep 11 13:46:07.908769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:07.908665    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" podStartSLOduration=1.908647966 podStartE2EDuration="1.908647966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:46:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:46:07.907871857 +0000 UTC m=+1492.301559268" watchObservedRunningTime="2026-09-11 13:46:07.908647966 +0000 UTC m=+1492.302335425"
Sep 11 13:46:08.977732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:08.977700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:10.228168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:10.228136    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.211680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.211639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.248840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.248813    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"]
Sep 11 13:46:16.249702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.249682    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-c0f589fb-29818905-xn7s6"]
Sep 11 13:46:16.294922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.294893    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:46:16.295130 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.295102    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" podUID="f33ffca4-8f97-463e-8135-b3036dd68491" containerName="payload-processing" containerID="cri-o://e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a" gracePeriod=30
Sep 11 13:46:16.349608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.349558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.428806 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.428773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.581080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.581010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.589289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.589262    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:46:16.591067 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.591027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:16.591340 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.591085    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:16.610011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.609980    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:16.653392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.653360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:16.664530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.664486    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume\") pod \"f33ffca4-8f97-463e-8135-b3036dd68491\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") "
Sep 11 13:46:16.664662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.664560    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bszrp\" (UniqueName: \"kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp\") pod \"f33ffca4-8f97-463e-8135-b3036dd68491\" (UID: \"f33ffca4-8f97-463e-8135-b3036dd68491\") "
Sep 11 13:46:16.664936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.664914    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f33ffca4-8f97-463e-8135-b3036dd68491" (UID: "f33ffca4-8f97-463e-8135-b3036dd68491"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:46:16.666674 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.666653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp" (OuterVolumeSpecName: "kube-api-access-bszrp") pod "f33ffca4-8f97-463e-8135-b3036dd68491" (UID: "f33ffca4-8f97-463e-8135-b3036dd68491"). InnerVolumeSpecName "kube-api-access-bszrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:46:16.672876 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.672545    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"]
Sep 11 13:46:16.674881 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.674849    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-c0f589fb-cfj78"]
Sep 11 13:46:16.766085 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.766059    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f33ffca4-8f97-463e-8135-b3036dd68491-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:46:16.766241 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.766088    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bszrp\" (UniqueName: \"kubernetes.io/projected/f33ffca4-8f97-463e-8135-b3036dd68491-kube-api-access-bszrp\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:46:16.928257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.928221    2576 generic.go:356] "Generic (PLEG): container finished" podID="f33ffca4-8f97-463e-8135-b3036dd68491" containerID="e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a" exitCode=0
Sep 11 13:46:16.928426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.928273    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"
Sep 11 13:46:16.928426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.928310    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" event={"ID":"f33ffca4-8f97-463e-8135-b3036dd68491","Type":"ContainerDied","Data":"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"}
Sep 11 13:46:16.928426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.928365    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5" event={"ID":"f33ffca4-8f97-463e-8135-b3036dd68491","Type":"ContainerDied","Data":"efd59eb3387d492c2ae4d41266e720593885c72a60619c6c71857f7bb01489b7"}
Sep 11 13:46:16.928426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.928392    2576 scope.go:117] "RemoveContainer" containerID="e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"
Sep 11 13:46:16.938998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.938979    2576 scope.go:117] "RemoveContainer" containerID="e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"
Sep 11 13:46:16.939267 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:46:16.939243    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a\": container with ID starting with e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a not found: ID does not exist" containerID="e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"
Sep 11 13:46:16.939321 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.939279    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a"} err="failed to get container status \"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a\": rpc error: code = NotFound desc = could not find container \"e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a\": container with ID starting with e7cdc2d355a51f19b55a45938404041774772e1f3f18aee513b2a278dc79461a not found: ID does not exist"
Sep 11 13:46:16.942309 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.942288    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:46:16.955435 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.955411    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:46:16.959541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:16.959521    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-c0f589fb-785b64bc5c-wh2v5"]
Sep 11 13:46:17.299744 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:17.299713    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:18.248636 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:18.248608    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47223586-fba3-4ff3-97a7-20cac4d537d3" path="/var/lib/kubelet/pods/47223586-fba3-4ff3-97a7-20cac4d537d3/volumes"
Sep 11 13:46:18.248937 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:18.248924    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55573de4-6435-402a-8de0-b90748a3d7dd" path="/var/lib/kubelet/pods/55573de4-6435-402a-8de0-b90748a3d7dd/volumes"
Sep 11 13:46:18.249264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:18.249252    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f33ffca4-8f97-463e-8135-b3036dd68491" path="/var/lib/kubelet/pods/f33ffca4-8f97-463e-8135-b3036dd68491/volumes"
Sep 11 13:46:18.377107 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:18.377058    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:19.626706 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:19.626678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:20.020603 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:20.020572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:20.704169 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:20.704130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:21.833514 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:21.833464    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:23.186904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:23.186872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:23.696724 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:23.696694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:24.418795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:24.418760    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:25.143635 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:25.143606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:44.076749 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:44.076666    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:46:46.551722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:46:46.551694    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:01.353928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:01.353896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:03.870922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:03.870892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:05.770227 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:05.770190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:11.786779 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:11.786744    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:13.900770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:13.900738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:13.999360 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:13.999330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:14.226208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:14.226129    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:47:14.226377 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:14.226355    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" containerID="cri-o://f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0" gracePeriod=30
Sep 11 13:47:14.273776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:14.273750    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:15.648736 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:15.648703    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:15.730548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:15.730518    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:15.832378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:15.832349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:16.266679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:16.266650    2576 scope.go:117] "RemoveContainer" containerID="1c9d6a291e4854a0740eccf436ba97717ae21231c5efc5612702c1cc24090e0a"
Sep 11 13:47:16.274600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:16.274585    2576 scope.go:117] "RemoveContainer" containerID="23dd5567c3b5d0b12bdfaca47a1de518f410650956b11afa105f55f6483e5898"
Sep 11 13:47:16.569326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:16.569255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:16.659802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:16.659773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:17.194640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:17.194606    2576 patch_prober.go:28] interesting pod/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:47:17.194835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:17.194667    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:47:18.180316 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:18.180284    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:19.480341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.480318    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:47:19.535677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535656    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.535803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535688    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2g2l\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.535803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535712    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.535803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535730    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.535803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535747    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.536021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.536021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535863    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.536021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.536021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535935    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert\") pod \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\" (UID: \"78cb895b-6ec1-4636-9ff1-a4b3bae05498\") "
Sep 11 13:47:19.536021 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.535973    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:47:19.536302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:47:19.536302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536184    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data" (OuterVolumeSpecName: "istio-data") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:47:19.536415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536361    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.536415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536379    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:47:19.536415 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536385    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.536640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536413    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:47:19.536640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.536421    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.537846 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.537825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l" (OuterVolumeSpecName: "kube-api-access-q2g2l") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "kube-api-access-q2g2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:47:19.537961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.537951    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:47:19.538019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.538005    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token" (OuterVolumeSpecName: "istio-token") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:47:19.538076 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.538064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78cb895b-6ec1-4636-9ff1-a4b3bae05498" (UID: "78cb895b-6ec1-4636-9ff1-a4b3bae05498"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:47:19.636997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.636968    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2g2l\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-kube-api-access-q2g2l\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.636997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.636994    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.637152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.637008    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.637152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.637028    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.637152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.637039    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb895b-6ec1-4636-9ff1-a4b3bae05498-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:19.637152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:19.637050    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb895b-6ec1-4636-9ff1-a4b3bae05498-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:47:20.247711 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.247679    2576 generic.go:356] "Generic (PLEG): container finished" podID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerID="f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0" exitCode=0
Sep 11 13:47:20.247902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.247714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerDied","Data":"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"}
Sep 11 13:47:20.247902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.247757    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp" event={"ID":"78cb895b-6ec1-4636-9ff1-a4b3bae05498","Type":"ContainerDied","Data":"bc0e24dbcd61b1805f00cb50fb79f095fbcfdc6dc4e0bb5838579d6f7f1b482b"}
Sep 11 13:47:20.247902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.247781    2576 scope.go:117] "RemoveContainer" containerID="f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"
Sep 11 13:47:20.247902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.247791    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"
Sep 11 13:47:20.257588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.257564    2576 scope.go:117] "RemoveContainer" containerID="b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"
Sep 11 13:47:20.265662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.265647    2576 scope.go:117] "RemoveContainer" containerID="f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"
Sep 11 13:47:20.265910 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:47:20.265890    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0\": container with ID starting with f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0 not found: ID does not exist" containerID="f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"
Sep 11 13:47:20.265975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.265916    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0"} err="failed to get container status \"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0\": rpc error: code = NotFound desc = could not find container \"f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0\": container with ID starting with f81e38f380075d6a5a21c43f8057a2ab0703e0d1bc678ae2c0c4ac2a9dffcdf0 not found: ID does not exist"
Sep 11 13:47:20.265975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.265930    2576 scope.go:117] "RemoveContainer" containerID="b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"
Sep 11 13:47:20.266164 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:47:20.266149    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12\": container with ID starting with b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12 not found: ID does not exist" containerID="b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"
Sep 11 13:47:20.266209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.266168    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12"} err="failed to get container status \"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12\": rpc error: code = NotFound desc = could not find container \"b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12\": container with ID starting with b1c1791a3854550fbf5dd4e3d412579cde2fb33187f0de3d9c1353a5b8328e12 not found: ID does not exist"
Sep 11 13:47:20.282301 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.282278    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:47:20.284006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:20.283989    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c0f589fb-openshift-default-687bb9b7b4-dswkp"]
Sep 11 13:47:21.109049 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:21.109005    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:22.248817 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:22.248787    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" path="/var/lib/kubelet/pods/78cb895b-6ec1-4636-9ff1-a4b3bae05498/volumes"
Sep 11 13:47:24.783875 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:24.783843    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:24.849013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:24.848986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:25.646359 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:25.646325    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:25.717334 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:25.717304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:26.389152 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:26.389124    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:28.137913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:28.137864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:36.091596 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:36.091566    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:36.375519 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:36.375431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:36.491389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:36.491359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:42.119419 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:42.119382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:46.893513 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:46.893470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:47.129153 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:47.129126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:47:48.107381 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:47:48.107350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:04.108324 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.108289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:04.211713 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.211680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:04.859206 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859177    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:48:04.859615 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859599    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:04.859675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859618    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:04.859675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859633    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f33ffca4-8f97-463e-8135-b3036dd68491" containerName="payload-processing"
Sep 11 13:48:04.859675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859639    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f33ffca4-8f97-463e-8135-b3036dd68491" containerName="payload-processing"
Sep 11 13:48:04.859776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859709    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:04.859776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859720    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:04.859776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.859728    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f33ffca4-8f97-463e-8135-b3036dd68491" containerName="payload-processing"
Sep 11 13:48:04.862696 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.862679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.876341 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.876321    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"kube-root-ca.crt\""
Sep 11 13:48:04.876452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.876338    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"openshift-service-ca.crt\""
Sep 11 13:48:04.876452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.876377    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"default-dockercfg-xvr7g\""
Sep 11 13:48:04.876568 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.876448    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-371c38\"/\"distinct-2-w4-371c38-kserve-self-signed-certs\""
Sep 11 13:48:04.880218 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.880199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:04.896722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.896703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:48:04.924864 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.924843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.924955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.924877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.924955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.924921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.925038 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.924965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.925038 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.924985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5g67\" (UniqueName: \"kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:04.925038 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:04.925003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.025730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.025698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.025907 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.025750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.025907 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.025775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5g67\" (UniqueName: \"kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.025907 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.025797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.026080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.025975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.026080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.026036    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.026176 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.026152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.026260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.026238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.026371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.026346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.028106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.028087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.028460 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.028440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.039123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.039098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5g67\" (UniqueName: \"kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67\") pod \"distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") " pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.171591 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.171490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:05.464192 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.464165    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:48:05.466696 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:48:05.466669    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42292481_a10b_4d13_8afe_8056569d3cc5.slice/crio-cb905a92753729459b3b09ac00093fa401c3b53cb62feafe01e149474965cbc1 WatchSource:0}: Error finding container cb905a92753729459b3b09ac00093fa401c3b53cb62feafe01e149474965cbc1: Status 404 returned error can't find the container with id cb905a92753729459b3b09ac00093fa401c3b53cb62feafe01e149474965cbc1
Sep 11 13:48:05.480369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:05.480337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" event={"ID":"42292481-a10b-4d13-8afe-8056569d3cc5","Type":"ContainerStarted","Data":"cb905a92753729459b3b09ac00093fa401c3b53cb62feafe01e149474965cbc1"}
Sep 11 13:48:06.487703 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:06.487665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" event={"ID":"42292481-a10b-4d13-8afe-8056569d3cc5","Type":"ContainerStarted","Data":"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"}
Sep 11 13:48:06.525867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:06.525813    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" podStartSLOduration=2.525797411 podStartE2EDuration="2.525797411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:06.523149682 +0000 UTC m=+1610.916837092" watchObservedRunningTime="2026-09-11 13:48:06.525797411 +0000 UTC m=+1610.919484859"
Sep 11 13:48:08.220356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:08.220323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:15.172611 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:15.172573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:15.172611 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:15.172620    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:15.185113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:15.185084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:15.547948 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:15.547922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:48:16.070329 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.070295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:16.173289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.173251    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:16.471225 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.471151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:16.570422 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.570392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:16.996293 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.996262    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:48:16.996644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.996629    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:16.996644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.996645    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cb895b-6ec1-4636-9ff1-a4b3bae05498" containerName="istio-proxy"
Sep 11 13:48:16.999616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:16.999600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.002005 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:48:17.001966    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-api-a-0a2395-openshift-default-dockercfg-b4ld6\" is forbidden: User \"system:node:ip-10-0-133-221.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-221.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0a2395-openshift-default-dockercfg-b4ld6\"" type="*v1.Secret"
Sep 11 13:48:17.015441 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:48:17.015417    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" is forbidden: User \"system:node:ip-10-0-133-221.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-133-221.ec2.internal' and this object" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.048039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.048015    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:48:17.060855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.060835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:17.141229 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141271    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141347 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk9n2\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141473    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.141653 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.141540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.195909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.195876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:17.242059 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk9n2\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242449 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.242691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242667    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.243005 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.242985    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.244741 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.244723    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.244893 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.244877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.262352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.262295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:17.263300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:17.263281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk9n2\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2\") pod \"e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") " pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:18.255282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.255245    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0a2395-openshift-default-dockercfg-b4ld6\""
Sep 11 13:48:18.259186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.259165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:18.611457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.611432    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:48:18.613327 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:48:18.613302    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b94392_91c7_4ac0_aaef_e535f91502a7.slice/crio-ca7b817336bb53b182bc19fb886a2afe3f6bd2a776c103458c23bee7dd1edf4a WatchSource:0}: Error finding container ca7b817336bb53b182bc19fb886a2afe3f6bd2a776c103458c23bee7dd1edf4a: Status 404 returned error can't find the container with id ca7b817336bb53b182bc19fb886a2afe3f6bd2a776c103458c23bee7dd1edf4a
Sep 11 13:48:18.615466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.615438    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:48:18.615560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.615518    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:48:18.615560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:18.615549    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:48:19.557389 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:19.557354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerStarted","Data":"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"}
Sep 11 13:48:19.557787 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:19.557403    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerStarted","Data":"ca7b817336bb53b182bc19fb886a2afe3f6bd2a776c103458c23bee7dd1edf4a"}
Sep 11 13:48:19.596026 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:19.595975    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podStartSLOduration=3.595960341 podStartE2EDuration="3.595960341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:19.594362523 +0000 UTC m=+1623.988049934" watchObservedRunningTime="2026-09-11 13:48:19.595960341 +0000 UTC m=+1623.989647752"
Sep 11 13:48:20.260245 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:20.260216    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:20.265753 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:20.265728    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:20.562818 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:20.562790    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:20.563698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:20.563681    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:48:21.106044 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:21.106015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:21.913439 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:21.913410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:22.038680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:22.038651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:23.549070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:23.549035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:26.013696 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:26.013663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:28.532677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:28.532644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:30.448961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:30.448911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:32.151402 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:32.151365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:33.923292 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:33.923256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:35.835413 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:35.835384    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:35.993064 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:35.993029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:36.305528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:36.305482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:36.913658 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:36.913628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:37.081869 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.081839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:37.443623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.443595    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:48:37.447221 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.447201    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.459629 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.459607    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-0a2395\""
Sep 11 13:48:37.459719 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.459632    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-0a2395-dockercfg-2p954\""
Sep 11 13:48:37.472932 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.472910    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:48:37.476198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.476180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.480526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.480479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:48:37.531168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.531137    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:48:37.535222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.535193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.535362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.535260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtpz\" (UniqueName: \"kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.535362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.535293    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.535362 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.535336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsn5x\" (UniqueName: \"kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.636609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.636579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsn5x\" (UniqueName: \"kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.636772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.636636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.636772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.636697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtpz\" (UniqueName: \"kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.636772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.636716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.637608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.637584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.637691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.637595    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.658120 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.658088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsn5x\" (UniqueName: \"kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x\") pod \"payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.659749 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.659728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtpz\" (UniqueName: \"kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz\") pod \"payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") " pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:37.756358 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.756303    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:37.787430 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:37.787406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:38.041621 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.041564    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:48:38.047907 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:48:38.047870    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2fb5f0_3a44_4e7f_a775_807a694b6184.slice/crio-d68be9ce6b743f0d59b37446d9bd03dafaf2eea7eebf2c2942c9d05e013aef78 WatchSource:0}: Error finding container d68be9ce6b743f0d59b37446d9bd03dafaf2eea7eebf2c2942c9d05e013aef78: Status 404 returned error can't find the container with id d68be9ce6b743f0d59b37446d9bd03dafaf2eea7eebf2c2942c9d05e013aef78
Sep 11 13:48:38.092297 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.092272    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:48:38.094262 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:48:38.094233    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8981c63_1683_41fa_bc2a_988b4c8f3b3a.slice/crio-4603a05c08f9fd1c618370df75d8d65179f020f251deaf1d4e6494d04276b56d WatchSource:0}: Error finding container 4603a05c08f9fd1c618370df75d8d65179f020f251deaf1d4e6494d04276b56d: Status 404 returned error can't find the container with id 4603a05c08f9fd1c618370df75d8d65179f020f251deaf1d4e6494d04276b56d
Sep 11 13:48:38.658778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.658740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" event={"ID":"f8981c63-1683-41fa-bc2a-988b4c8f3b3a","Type":"ContainerStarted","Data":"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"}
Sep 11 13:48:38.658985 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.658803    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" event={"ID":"f8981c63-1683-41fa-bc2a-988b4c8f3b3a","Type":"ContainerStarted","Data":"4603a05c08f9fd1c618370df75d8d65179f020f251deaf1d4e6494d04276b56d"}
Sep 11 13:48:38.658985 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.658836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:38.660465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.660434    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" event={"ID":"dc2fb5f0-3a44-4e7f-a775-807a694b6184","Type":"ContainerStarted","Data":"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"}
Sep 11 13:48:38.660579 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.660483    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" event={"ID":"dc2fb5f0-3a44-4e7f-a775-807a694b6184","Type":"ContainerStarted","Data":"d68be9ce6b743f0d59b37446d9bd03dafaf2eea7eebf2c2942c9d05e013aef78"}
Sep 11 13:48:38.660579 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.660549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:38.689761 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.689712    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" podStartSLOduration=1.689694692 podStartE2EDuration="1.689694692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:38.688939997 +0000 UTC m=+1643.082627420" watchObservedRunningTime="2026-09-11 13:48:38.689694692 +0000 UTC m=+1643.083382118"
Sep 11 13:48:38.732645 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:38.732599    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" podStartSLOduration=1.73258108 podStartE2EDuration="1.73258108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:48:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:48:38.730525673 +0000 UTC m=+1643.124213083" watchObservedRunningTime="2026-09-11 13:48:38.73258108 +0000 UTC m=+1643.126268492"
Sep 11 13:48:49.673299 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:49.673263    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:48:49.673777 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:49.673340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:48:51.385356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:51.385317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:48:59.760916 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:48:59.760880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:00.722595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:00.722561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:01.801953 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:01.801922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:02.290399 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:02.290367    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:02.714826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:02.714742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:02.838800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:02.838771    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.262012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:03.261977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.575978 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:03.575948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:03.994972 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:03.994888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:05.384260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:05.384228    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:06.509386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:06.509353    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:06.610265 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:06.610231    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:07.056610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:07.056576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:08.985285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:08.985246    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:10.537464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:10.537431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:10.644693 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:10.644657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:10.742650 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:10.742618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:11.328412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.328378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:11.445308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.445282    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:11.635396 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.635303    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:49:11.639721 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.639697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.654256 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.654234    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-c8d2f6-dockercfg-j8n8g\""
Sep 11 13:49:11.654462 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.654413    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-c8d2f6\""
Sep 11 13:49:11.668562 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.668530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.668883 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.668864    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4nsg\" (UniqueName: \"kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.669120 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.666801    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:49:11.770052 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.770017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4nsg\" (UniqueName: \"kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.770241 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.770117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.770832 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.770812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.782485 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.782455    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4nsg\" (UniqueName: \"kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg\") pod \"payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") " pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:11.968369 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:11.968272    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:12.144228 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.144195    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:12.286794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.286766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:49:12.289594 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:49:12.289561    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22f2fd5_0034_4997_8619_e984c31088ac.slice/crio-050d21744728c1087032d3a2c8545aa43342574aa0b34b3b3683df1d7c5bb73a WatchSource:0}: Error finding container 050d21744728c1087032d3a2c8545aa43342574aa0b34b3b3683df1d7c5bb73a: Status 404 returned error can't find the container with id 050d21744728c1087032d3a2c8545aa43342574aa0b34b3b3683df1d7c5bb73a
Sep 11 13:49:12.848110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.848075    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" event={"ID":"e22f2fd5-0034-4997-8619-e984c31088ac","Type":"ContainerStarted","Data":"e0c8007de7b65c3128b7e760d79ab0a4304f2671c73cb4ef744c5f37a1637af9"}
Sep 11 13:49:12.848524 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.848128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" event={"ID":"e22f2fd5-0034-4997-8619-e984c31088ac","Type":"ContainerStarted","Data":"050d21744728c1087032d3a2c8545aa43342574aa0b34b3b3683df1d7c5bb73a"}
Sep 11 13:49:12.848617 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.848597    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:12.872113 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:12.872065    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" podStartSLOduration=1.872046498 podStartE2EDuration="1.872046498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:49:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:49:12.870664339 +0000 UTC m=+1677.264351750" watchObservedRunningTime="2026-09-11 13:49:12.872046498 +0000 UTC m=+1677.265733941"
Sep 11 13:49:13.231836 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:13.231741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:14.286783 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:14.286743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:15.487805 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:15.487765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:16.311273 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.311243    2576 scope.go:117] "RemoveContainer" containerID="34c298d03872a43ce66549b64ed8d8b5f4bae7f50c30235e2f73b54ecf0aac73"
Sep 11 13:49:16.428625 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.428594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:16.871534 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.871427    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerID="c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee" exitCode=0
Sep 11 13:49:16.871534 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.871467    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerDied","Data":"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"}
Sep 11 13:49:16.872098 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.871973    2576 scope.go:117] "RemoveContainer" containerID="c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"
Sep 11 13:49:16.873234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.873203    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:49:16.873331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.873271    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:49:16.873331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:16.873310    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:49:17.316467 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:17.316434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:17.878555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:17.878520    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerStarted","Data":"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"}
Sep 11 13:49:18.260158 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:18.260082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:18.261003 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:18.260977    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:49:18.261146 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:18.261026    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 13:49:18.542330 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:18.542289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:19.260376 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:19.260341    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:49:19.260773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:19.260408    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 13:49:20.259539 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:20.259507    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:49:20.259704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:20.259569    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 13:49:20.339033 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:20.339004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:20.562928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:20.562883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:21.195942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:21.195912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:21.260105 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:21.260064    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:49:21.260294 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:21.260139    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 13:49:21.363250 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:21.363219    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:22.259730 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:22.259698    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:49:22.259900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:22.259747    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 11 13:49:22.417555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:22.417474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:23.264278 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:23.264249    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:23.265268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:23.265245    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:23.635594 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:23.635561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:24.669853 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:24.669824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:24.862921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:24.862889    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:49:25.787035 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:25.786992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:26.902691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:26.902657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:27.933300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:27.933269    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:29.032240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:29.032207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:29.393100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:29.392995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:29.493661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:29.493631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:29.914340 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:29.914310    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:30.900762 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:30.900726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:31.803025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:31.802997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:32.967322 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:32.967297    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:34.185925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:34.185887    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:35.479485 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:35.479433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:41.897575 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:41.897435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:41.959737 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:41.959703    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:49:41.960190 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:41.960159    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" podUID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" containerName="payload-processing" containerID="cri-o://710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941" gracePeriod=30
Sep 11 13:49:41.966400 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:41.966187    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:49:41.966471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:41.966399    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" podUID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" containerName="payload-pre-processing" containerID="cri-o://0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda" gracePeriod=30
Sep 11 13:49:42.013252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.013225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:42.113902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.113870    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:42.217913 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.217752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:42.245292 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.245247    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:49:42.276800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.276767    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:49:42.302492 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.302454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:42.380427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.380395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsn5x\" (UniqueName: \"kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x\") pod \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") "
Sep 11 13:49:42.380618 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.380467    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdtpz\" (UniqueName: \"kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz\") pod \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") "
Sep 11 13:49:42.380618 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.380558    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume\") pod \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\" (UID: \"dc2fb5f0-3a44-4e7f-a775-807a694b6184\") "
Sep 11 13:49:42.380618 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.380589    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume\") pod \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\" (UID: \"f8981c63-1683-41fa-bc2a-988b4c8f3b3a\") "
Sep 11 13:49:42.380918 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.380894    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc2fb5f0-3a44-4e7f-a775-807a694b6184" (UID: "dc2fb5f0-3a44-4e7f-a775-807a694b6184"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:49:42.381131 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.381104    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f8981c63-1683-41fa-bc2a-988b4c8f3b3a" (UID: "f8981c63-1683-41fa-bc2a-988b4c8f3b3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:49:42.382654 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.382627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz" (OuterVolumeSpecName: "kube-api-access-tdtpz") pod "f8981c63-1683-41fa-bc2a-988b4c8f3b3a" (UID: "f8981c63-1683-41fa-bc2a-988b4c8f3b3a"). InnerVolumeSpecName "kube-api-access-tdtpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:49:42.382746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.382656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x" (OuterVolumeSpecName: "kube-api-access-hsn5x") pod "dc2fb5f0-3a44-4e7f-a775-807a694b6184" (UID: "dc2fb5f0-3a44-4e7f-a775-807a694b6184"). InnerVolumeSpecName "kube-api-access-hsn5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:49:42.404928 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.404897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:42.481688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.481612    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsn5x\" (UniqueName: \"kubernetes.io/projected/dc2fb5f0-3a44-4e7f-a775-807a694b6184-kube-api-access-hsn5x\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:42.481688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.481637    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdtpz\" (UniqueName: \"kubernetes.io/projected/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-kube-api-access-tdtpz\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:42.481688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.481647    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc2fb5f0-3a44-4e7f-a775-807a694b6184-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:42.481688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.481658    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f8981c63-1683-41fa-bc2a-988b4c8f3b3a-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:42.717623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:42.717593    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:43.007398 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.007364    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" containerID="0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda" exitCode=0
Sep 11 13:49:43.007813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.007415    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"
Sep 11 13:49:43.007813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.007449    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" event={"ID":"dc2fb5f0-3a44-4e7f-a775-807a694b6184","Type":"ContainerDied","Data":"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"}
Sep 11 13:49:43.007813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.007541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685" event={"ID":"dc2fb5f0-3a44-4e7f-a775-807a694b6184","Type":"ContainerDied","Data":"d68be9ce6b743f0d59b37446d9bd03dafaf2eea7eebf2c2942c9d05e013aef78"}
Sep 11 13:49:43.007813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.007566    2576 scope.go:117] "RemoveContainer" containerID="0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"
Sep 11 13:49:43.008903 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.008884    2576 generic.go:356] "Generic (PLEG): container finished" podID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" containerID="710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941" exitCode=0
Sep 11 13:49:43.008975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.008947    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"
Sep 11 13:49:43.008975 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.008954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" event={"ID":"f8981c63-1683-41fa-bc2a-988b4c8f3b3a","Type":"ContainerDied","Data":"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"}
Sep 11 13:49:43.009173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.008983    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2" event={"ID":"f8981c63-1683-41fa-bc2a-988b4c8f3b3a","Type":"ContainerDied","Data":"4603a05c08f9fd1c618370df75d8d65179f020f251deaf1d4e6494d04276b56d"}
Sep 11 13:49:43.016201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.016185    2576 scope.go:117] "RemoveContainer" containerID="0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"
Sep 11 13:49:43.016536 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:49:43.016481    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda\": container with ID starting with 0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda not found: ID does not exist" containerID="0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"
Sep 11 13:49:43.016638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.016547    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda"} err="failed to get container status \"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda\": rpc error: code = NotFound desc = could not find container \"0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda\": container with ID starting with 0a696c54783b3e20634cfda1f27ad9b1d79ada69fcd5b51ad7dd0644ce6c4fda not found: ID does not exist"
Sep 11 13:49:43.016638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.016571    2576 scope.go:117] "RemoveContainer" containerID="710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"
Sep 11 13:49:43.024573 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.024558    2576 scope.go:117] "RemoveContainer" containerID="710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"
Sep 11 13:49:43.024805 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:49:43.024789    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941\": container with ID starting with 710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941 not found: ID does not exist" containerID="710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"
Sep 11 13:49:43.024856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.024810    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941"} err="failed to get container status \"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941\": rpc error: code = NotFound desc = could not find container \"710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941\": container with ID starting with 710bc7870a737c35c7106d83f6b6a08c98c1ac6f33b6d1c53e319ffa2b85d941 not found: ID does not exist"
Sep 11 13:49:43.041807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.041783    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:49:43.046573 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.046554    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0a2395-6bf555c9bb-b8685"]
Sep 11 13:49:43.061254 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.061234    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:49:43.067220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.067201    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0a2395-ff79cdc75-t5hw2"]
Sep 11 13:49:43.575446 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:43.575409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:44.249554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:44.249512    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" path="/var/lib/kubelet/pods/dc2fb5f0-3a44-4e7f-a775-807a694b6184/volumes"
Sep 11 13:49:44.250084 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:44.250062    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" path="/var/lib/kubelet/pods/f8981c63-1683-41fa-bc2a-988b4c8f3b3a/volumes"
Sep 11 13:49:44.585132 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:44.585099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:45.864545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:45.864490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:47.258797 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:47.258765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:48.317716 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:48.317686    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:49.552945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:49.552911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:50.201257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.201217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:50.272046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.272017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:50.533532 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.533488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:50.669632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.669596    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:49:50.670019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.669797    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" containerID="cri-o://ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f" gracePeriod=30
Sep 11 13:49:50.671173 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:50.671147    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:51.694795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:51.694766    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:52.656981 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:52.656946    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:53.267191 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:53.267153    2576 patch_prober.go:28] interesting pod/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:49:53.267587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:53.267221    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:49:53.818083 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:53.818047    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:54.705427 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:54.705392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:55.395426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:55.395394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:55.476760 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:55.476719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:55.676226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:55.676129    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:55.924849 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:55.924825    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:56.005274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005193    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005238    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005274 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005267    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005307    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005330    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005348    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005367    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005390    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005415    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk9n2\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2\") pod \"a7b94392-91c7-4ac0-aaef-e535f91502a7\" (UID: \"a7b94392-91c7-4ac0-aaef-e535f91502a7\") "
Sep 11 13:49:56.005867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005601    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:49:56.005867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005632    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:49:56.005867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005834    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:49:56.006046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005858    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:49:56.006046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005901    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.006046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005919    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.006046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.005964    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:49:56.007646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.007612    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:49:56.007749 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.007740    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2" (OuterVolumeSpecName: "kube-api-access-lk9n2") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "kube-api-access-lk9n2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:49:56.007947 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.007928    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:49:56.007947 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.007932    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "a7b94392-91c7-4ac0-aaef-e535f91502a7" (UID: "a7b94392-91c7-4ac0-aaef-e535f91502a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:49:56.076945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.076911    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerID="ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f" exitCode=0
Sep 11 13:49:56.077137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.076974    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"
Sep 11 13:49:56.077137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.076996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerDied","Data":"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"}
Sep 11 13:49:56.077137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.077054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn" event={"ID":"a7b94392-91c7-4ac0-aaef-e535f91502a7","Type":"ContainerDied","Data":"ca7b817336bb53b182bc19fb886a2afe3f6bd2a776c103458c23bee7dd1edf4a"}
Sep 11 13:49:56.077137 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.077094    2576 scope.go:117] "RemoveContainer" containerID="ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"
Sep 11 13:49:56.087317 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.087301    2576 scope.go:117] "RemoveContainer" containerID="c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"
Sep 11 13:49:56.097203 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.097180    2576 scope.go:117] "RemoveContainer" containerID="ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"
Sep 11 13:49:56.097491 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:49:56.097472    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f\": container with ID starting with ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f not found: ID does not exist" containerID="ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"
Sep 11 13:49:56.097583 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.097514    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f"} err="failed to get container status \"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f\": rpc error: code = NotFound desc = could not find container \"ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f\": container with ID starting with ea1c563e5ba1278a9118be5f0f645fc195fbc4b1e8dbe3c2c0e083ab95d64d3f not found: ID does not exist"
Sep 11 13:49:56.097583 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.097533    2576 scope.go:117] "RemoveContainer" containerID="c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"
Sep 11 13:49:56.097803 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:49:56.097785    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee\": container with ID starting with c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee not found: ID does not exist" containerID="c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"
Sep 11 13:49:56.097850 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.097809    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee"} err="failed to get container status \"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee\": rpc error: code = NotFound desc = could not find container \"c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee\": container with ID starting with c632354193b53f6bc1898096b0765b575f7455e80b2df6ee376888d89a2bfaee not found: ID does not exist"
Sep 11 13:49:56.106365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106346    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106364    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106372    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7b94392-91c7-4ac0-aaef-e535f91502a7-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106380    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106390    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7b94392-91c7-4ac0-aaef-e535f91502a7-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106398    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7b94392-91c7-4ac0-aaef-e535f91502a7-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.106489 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.106406    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk9n2\" (UniqueName: \"kubernetes.io/projected/a7b94392-91c7-4ac0-aaef-e535f91502a7-kube-api-access-lk9n2\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:49:56.118840 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.118816    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:49:56.122608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.122586    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0a2395-openshift-default-69d969d9bd-w4kfn"]
Sep 11 13:49:56.248626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.248595    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" path="/var/lib/kubelet/pods/a7b94392-91c7-4ac0-aaef-e535f91502a7/volumes"
Sep 11 13:49:56.900745 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:56.900711    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:57.799731 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:57.799700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:49:58.928665 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:49:58.928625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:06.813550 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:06.813488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:06.936681 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:06.936647    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:50:06.936951 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:06.936905    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" podUID="e22f2fd5-0034-4997-8619-e984c31088ac" containerName="payload-processing" containerID="cri-o://e0c8007de7b65c3128b7e760d79ab0a4304f2671c73cb4ef744c5f37a1637af9" gracePeriod=30
Sep 11 13:50:06.978785 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:06.978757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:07.081070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.080952    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:07.136949 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.136913    2576 generic.go:356] "Generic (PLEG): container finished" podID="e22f2fd5-0034-4997-8619-e984c31088ac" containerID="e0c8007de7b65c3128b7e760d79ab0a4304f2671c73cb4ef744c5f37a1637af9" exitCode=0
Sep 11 13:50:07.137133 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.136984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" event={"ID":"e22f2fd5-0034-4997-8619-e984c31088ac","Type":"ContainerDied","Data":"e0c8007de7b65c3128b7e760d79ab0a4304f2671c73cb4ef744c5f37a1637af9"}
Sep 11 13:50:07.188031 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.187969    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:07.213445 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.213422    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:50:07.260464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.260434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:07.308812 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.308784    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume\") pod \"e22f2fd5-0034-4997-8619-e984c31088ac\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") "
Sep 11 13:50:07.308965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.308881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4nsg\" (UniqueName: \"kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg\") pod \"e22f2fd5-0034-4997-8619-e984c31088ac\" (UID: \"e22f2fd5-0034-4997-8619-e984c31088ac\") "
Sep 11 13:50:07.309243 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.309215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e22f2fd5-0034-4997-8619-e984c31088ac" (UID: "e22f2fd5-0034-4997-8619-e984c31088ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:07.311039 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.311016    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg" (OuterVolumeSpecName: "kube-api-access-z4nsg") pod "e22f2fd5-0034-4997-8619-e984c31088ac" (UID: "e22f2fd5-0034-4997-8619-e984c31088ac"). InnerVolumeSpecName "kube-api-access-z4nsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:07.410772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.410664    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e22f2fd5-0034-4997-8619-e984c31088ac-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:07.410772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.410701    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4nsg\" (UniqueName: \"kubernetes.io/projected/e22f2fd5-0034-4997-8619-e984c31088ac-kube-api-access-z4nsg\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:07.713068 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.712970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:07.928099 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:07.928071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:08.143392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.143358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj" event={"ID":"e22f2fd5-0034-4997-8619-e984c31088ac","Type":"ContainerDied","Data":"050d21744728c1087032d3a2c8545aa43342574aa0b34b3b3683df1d7c5bb73a"}
Sep 11 13:50:08.143392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.143377    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"
Sep 11 13:50:08.143686 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.143414    2576 scope.go:117] "RemoveContainer" containerID="e0c8007de7b65c3128b7e760d79ab0a4304f2671c73cb4ef744c5f37a1637af9"
Sep 11 13:50:08.172234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.172206    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:50:08.177865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.177843    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-c8d2f6-5c9bcfd567-bjrbj"]
Sep 11 13:50:08.248339 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:08.248312    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e22f2fd5-0034-4997-8619-e984c31088ac" path="/var/lib/kubelet/pods/e22f2fd5-0034-4997-8619-e984c31088ac/volumes"
Sep 11 13:50:09.263683 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:09.263652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:10.883434 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:10.883402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:12.048040 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:12.048006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:13.243424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:13.243392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:14.305664 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:14.305638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:15.304630 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:15.304595    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:15.390335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:15.390304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:15.487691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:15.487659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:15.696704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:15.696616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:16.186634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.186584    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"]
Sep 11 13:50:16.187352 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187328    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e22f2fd5-0034-4997-8619-e984c31088ac" containerName="payload-processing"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187354    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e22f2fd5-0034-4997-8619-e984c31088ac" containerName="payload-processing"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187405    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187414    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187434    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" containerName="payload-pre-processing"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187444    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" containerName="payload-pre-processing"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187463    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:16.187470 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187471    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187508    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" containerName="payload-processing"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187517    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" containerName="payload-processing"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187670    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2fb5f0-3a44-4e7f-a775-807a694b6184" containerName="payload-pre-processing"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187683    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8981c63-1683-41fa-bc2a-988b4c8f3b3a" containerName="payload-processing"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187699    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:16.187841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.187710    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e22f2fd5-0034-4997-8619-e984c31088ac" containerName="payload-processing"
Sep 11 13:50:16.193194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.193173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:16.196087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.196070    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:50:16.196288 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.196276    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:50:16.209245 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.209222    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"]
Sep 11 13:50:16.299136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.299104    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2\" (UID: \"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:16.399626 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.399603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2\" (UID: \"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:16.400168 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.400150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2\" (UID: \"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:16.503486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.503408    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:16.781233 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:16.781202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"]
Sep 11 13:50:16.784094 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:50:16.784069    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb45f5f_519b_4f1f_80ce_47a8cdcc8d24.slice/crio-d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860 WatchSource:0}: Error finding container d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860: Status 404 returned error can't find the container with id d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860
Sep 11 13:50:17.064570 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:17.064524    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:17.197722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:17.197688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2" event={"ID":"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24","Type":"ContainerStarted","Data":"3444292794fd85b21df84166fa40a4d369727baa705b99a481874b61b0549968"}
Sep 11 13:50:17.197900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:17.197738    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2" event={"ID":"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24","Type":"ContainerStarted","Data":"d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860"}
Sep 11 13:50:18.204392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:18.204361    2576 generic.go:356] "Generic (PLEG): container finished" podID="6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" containerID="3444292794fd85b21df84166fa40a4d369727baa705b99a481874b61b0549968" exitCode=0
Sep 11 13:50:18.204815 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:18.204406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2" event={"ID":"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24","Type":"ContainerDied","Data":"3444292794fd85b21df84166fa40a4d369727baa705b99a481874b61b0549968"}
Sep 11 13:50:18.371356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:18.371327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:19.334614 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:19.334592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:19.429466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:19.429435    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca\") pod \"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24\" (UID: \"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24\") "
Sep 11 13:50:19.429835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:19.429815    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" (UID: "6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:19.530429 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:19.530405    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:19.717201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:19.717171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:20.216124 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:20.216095    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2" event={"ID":"6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24","Type":"ContainerDied","Data":"d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860"}
Sep 11 13:50:20.216319 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:20.216145    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0be69b0309fb90bd37d72e1d9e469924db4dffed518f503eb686015f4c44860"
Sep 11 13:50:20.216319 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:20.216103    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"
Sep 11 13:50:20.250389 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:20.250363    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb45f5f_519b_4f1f_80ce_47a8cdcc8d24.slice\": RecentStats: unable to find data in memory cache]"
Sep 11 13:50:21.006218 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:21.006171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:21.976392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:21.976358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:23.096994 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:23.096960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:24.118942 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:24.118906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.103661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.103628    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.154337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.154306    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:50:27.154554 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.154519    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" podUID="c15f0cd5-785e-46bf-8905-e463b9e32535" containerName="payload-processing" containerID="cri-o://88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c" gracePeriod=30
Sep 11 13:50:27.174009 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.173984    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:50:27.174181 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.174163    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" podUID="ba205218-35c2-45d9-89f4-3a9318037148" containerName="payload-pre-processing" containerID="cri-o://23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89" gracePeriod=30
Sep 11 13:50:27.199876 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.199852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.300477 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.300451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.402748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.402374    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.427086 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.427014    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"]
Sep 11 13:50:27.429066 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.429034    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-470eeb-hrhj2"]
Sep 11 13:50:27.431943 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.431925    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:50:27.468705 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.468684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:27.476024 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.476004    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:50:27.616027 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.615992    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume\") pod \"ba205218-35c2-45d9-89f4-3a9318037148\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") "
Sep 11 13:50:27.616219 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.616039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume\") pod \"c15f0cd5-785e-46bf-8905-e463b9e32535\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") "
Sep 11 13:50:27.616219 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.616083    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln8h8\" (UniqueName: \"kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8\") pod \"ba205218-35c2-45d9-89f4-3a9318037148\" (UID: \"ba205218-35c2-45d9-89f4-3a9318037148\") "
Sep 11 13:50:27.616219 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.616128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5zhx\" (UniqueName: \"kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx\") pod \"c15f0cd5-785e-46bf-8905-e463b9e32535\" (UID: \"c15f0cd5-785e-46bf-8905-e463b9e32535\") "
Sep 11 13:50:27.616469 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.616428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ba205218-35c2-45d9-89f4-3a9318037148" (UID: "ba205218-35c2-45d9-89f4-3a9318037148"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:27.616571 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.616487    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c15f0cd5-785e-46bf-8905-e463b9e32535" (UID: "c15f0cd5-785e-46bf-8905-e463b9e32535"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:27.618209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.618186    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8" (OuterVolumeSpecName: "kube-api-access-ln8h8") pod "ba205218-35c2-45d9-89f4-3a9318037148" (UID: "ba205218-35c2-45d9-89f4-3a9318037148"). InnerVolumeSpecName "kube-api-access-ln8h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:27.618271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.618185    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx" (OuterVolumeSpecName: "kube-api-access-h5zhx") pod "c15f0cd5-785e-46bf-8905-e463b9e32535" (UID: "c15f0cd5-785e-46bf-8905-e463b9e32535"). InnerVolumeSpecName "kube-api-access-h5zhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:27.716996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.716913    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba205218-35c2-45d9-89f4-3a9318037148-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:27.716996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.716943    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15f0cd5-785e-46bf-8905-e463b9e32535-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:27.716996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.716952    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln8h8\" (UniqueName: \"kubernetes.io/projected/ba205218-35c2-45d9-89f4-3a9318037148-kube-api-access-ln8h8\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:27.716996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.716961    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5zhx\" (UniqueName: \"kubernetes.io/projected/c15f0cd5-785e-46bf-8905-e463b9e32535-kube-api-access-h5zhx\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:27.937585 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:27.937536    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:28.052711 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.052681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:28.250025 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.249994    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" path="/var/lib/kubelet/pods/6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24/volumes"
Sep 11 13:50:28.258068 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.258041    2576 generic.go:356] "Generic (PLEG): container finished" podID="c15f0cd5-785e-46bf-8905-e463b9e32535" containerID="88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c" exitCode=0
Sep 11 13:50:28.258209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.258083    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"
Sep 11 13:50:28.258209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.258121    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" event={"ID":"c15f0cd5-785e-46bf-8905-e463b9e32535","Type":"ContainerDied","Data":"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"}
Sep 11 13:50:28.258209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.258166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc" event={"ID":"c15f0cd5-785e-46bf-8905-e463b9e32535","Type":"ContainerDied","Data":"c05c17399d3a48df6ae047817afb20677e830dda5a5118a73742aa7eefa732f4"}
Sep 11 13:50:28.258209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.258194    2576 scope.go:117] "RemoveContainer" containerID="88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"
Sep 11 13:50:28.259921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.259899    2576 generic.go:356] "Generic (PLEG): container finished" podID="ba205218-35c2-45d9-89f4-3a9318037148" containerID="23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89" exitCode=0
Sep 11 13:50:28.260000 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.259929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" event={"ID":"ba205218-35c2-45d9-89f4-3a9318037148","Type":"ContainerDied","Data":"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"}
Sep 11 13:50:28.260000 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.259947    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"
Sep 11 13:50:28.260000 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.259955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn" event={"ID":"ba205218-35c2-45d9-89f4-3a9318037148","Type":"ContainerDied","Data":"0b932218a2ac87cd7cae1483575f9e0a4d30ac8641726c179146660312751ee7"}
Sep 11 13:50:28.267857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.267839    2576 scope.go:117] "RemoveContainer" containerID="88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"
Sep 11 13:50:28.268102 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:28.268085    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c\": container with ID starting with 88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c not found: ID does not exist" containerID="88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"
Sep 11 13:50:28.268160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.268109    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c"} err="failed to get container status \"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c\": rpc error: code = NotFound desc = could not find container \"88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c\": container with ID starting with 88475aed6dc8eb506aa9bb85421686498e12f75238965b2d514b443e547d245c not found: ID does not exist"
Sep 11 13:50:28.268160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.268125    2576 scope.go:117] "RemoveContainer" containerID="23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"
Sep 11 13:50:28.275579 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.275543    2576 scope.go:117] "RemoveContainer" containerID="23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"
Sep 11 13:50:28.275800 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:28.275782    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89\": container with ID starting with 23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89 not found: ID does not exist" containerID="23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"
Sep 11 13:50:28.275848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.275806    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89"} err="failed to get container status \"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89\": rpc error: code = NotFound desc = could not find container \"23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89\": container with ID starting with 23b62257f85a23614c036bb8eca3039d92e7c7f214494a1700d4bdec253a4e89 not found: ID does not exist"
Sep 11 13:50:28.289633 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.289610    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:50:28.294714 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.294694    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-470eeb-59567df55d-pp8qn"]
Sep 11 13:50:28.314267 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.314197    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:50:28.322764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:28.322733    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-470eeb-57759d79bc-sgblc"]
Sep 11 13:50:29.020856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:29.020824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:30.248945 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:30.248905    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba205218-35c2-45d9-89f4-3a9318037148" path="/var/lib/kubelet/pods/ba205218-35c2-45d9-89f4-3a9318037148/volumes"
Sep 11 13:50:30.249526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:30.249475    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15f0cd5-785e-46bf-8905-e463b9e32535" path="/var/lib/kubelet/pods/c15f0cd5-785e-46bf-8905-e463b9e32535/volumes"
Sep 11 13:50:30.976355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:30.976320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:32.141603 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:32.141565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:33.537271 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:33.537237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:34.773605 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:34.773572    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:35.822763 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:35.822728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:36.083917 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.083830    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"]
Sep 11 13:50:36.084175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084163    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba205218-35c2-45d9-89f4-3a9318037148" containerName="payload-pre-processing"
Sep 11 13:50:36.084226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084177    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba205218-35c2-45d9-89f4-3a9318037148" containerName="payload-pre-processing"
Sep 11 13:50:36.084226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084186    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15f0cd5-785e-46bf-8905-e463b9e32535" containerName="payload-processing"
Sep 11 13:50:36.084226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084192    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15f0cd5-785e-46bf-8905-e463b9e32535" containerName="payload-processing"
Sep 11 13:50:36.084226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084211    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" containerName="revoke-keys"
Sep 11 13:50:36.084226 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084219    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" containerName="revoke-keys"
Sep 11 13:50:36.084379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084271    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15f0cd5-785e-46bf-8905-e463b9e32535" containerName="payload-processing"
Sep 11 13:50:36.084379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084279    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb45f5f-519b-4f1f-80ce-47a8cdcc8d24" containerName="revoke-keys"
Sep 11 13:50:36.084379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084285    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba205218-35c2-45d9-89f4-3a9318037148" containerName="payload-pre-processing"
Sep 11 13:50:36.084379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.084292    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7b94392-91c7-4ac0-aaef-e535f91502a7" containerName="istio-proxy"
Sep 11 13:50:36.088804 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.088788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:36.094101 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.094078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:50:36.094328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.094086    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:50:36.110372 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.110344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"]
Sep 11 13:50:36.195240 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.195199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s\" (UID: \"0b68250e-2320-4ac5-8a33-072028530f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:36.296447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.296418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s\" (UID: \"0b68250e-2320-4ac5-8a33-072028530f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:36.297209 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.297188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s\" (UID: \"0b68250e-2320-4ac5-8a33-072028530f47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:36.398860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.398782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:36.555543 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.555488    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:36.676175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.676150    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"]
Sep 11 13:50:36.678012 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:50:36.677988    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b68250e_2320_4ac5_8a33_072028530f47.slice/crio-adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0 WatchSource:0}: Error finding container adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0: Status 404 returned error can't find the container with id adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0
Sep 11 13:50:36.895101 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:36.895072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:37.004471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.004377    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:37.306084 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.306042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s" event={"ID":"0b68250e-2320-4ac5-8a33-072028530f47","Type":"ContainerStarted","Data":"2f523f46ad22afdac342ae51bf99e60aa2b65809da344135b66695a579b21022"}
Sep 11 13:50:37.306300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.306107    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s" event={"ID":"0b68250e-2320-4ac5-8a33-072028530f47","Type":"ContainerStarted","Data":"adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0"}
Sep 11 13:50:37.387800 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.387745    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s" podStartSLOduration=1.387728037 podStartE2EDuration="1.387728037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:50:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:50:37.365646927 +0000 UTC m=+1761.759334338" watchObservedRunningTime="2026-09-11 13:50:37.387728037 +0000 UTC m=+1761.781415668"
Sep 11 13:50:37.388185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.388170    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:50:37.388367 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.388346    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy" containerID="cri-o://710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e" gracePeriod=30
Sep 11 13:50:37.412552 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:37.412519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:38.312778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:38.312740    2576 generic.go:356] "Generic (PLEG): container finished" podID="0b68250e-2320-4ac5-8a33-072028530f47" containerID="2f523f46ad22afdac342ae51bf99e60aa2b65809da344135b66695a579b21022" exitCode=0
Sep 11 13:50:38.313157 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:38.312815    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s" event={"ID":"0b68250e-2320-4ac5-8a33-072028530f47","Type":"ContainerDied","Data":"2f523f46ad22afdac342ae51bf99e60aa2b65809da344135b66695a579b21022"}
Sep 11 13:50:38.618648 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:38.618571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:39.452154 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:39.452125    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:39.527475 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:39.527438    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca\") pod \"0b68250e-2320-4ac5-8a33-072028530f47\" (UID: \"0b68250e-2320-4ac5-8a33-072028530f47\") "
Sep 11 13:50:39.527857 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:39.527833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0b68250e-2320-4ac5-8a33-072028530f47" (UID: "0b68250e-2320-4ac5-8a33-072028530f47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:39.629042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:39.628949    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b68250e-2320-4ac5-8a33-072028530f47-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:39.729969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:39.729935    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:40.324236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:40.324208    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"
Sep 11 13:50:40.324392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:40.324238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s" event={"ID":"0b68250e-2320-4ac5-8a33-072028530f47","Type":"ContainerDied","Data":"adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0"}
Sep 11 13:50:40.324392 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:40.324284    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adc20a1adf15fc5c9b64b37cfb006dd1df67d867218ae3217abb28c10e8700d0"
Sep 11 13:50:40.796861 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:40.796831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:41.037160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:41.037126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:42.166961 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.166934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:42.631835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.631811    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:50:42.757537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757434    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757474    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8wqv\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757537 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757528    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757570    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757665    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757698    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757735    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757752    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:42.757807 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757777    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token\") pod \"ef9db282-ccda-49e3-a271-934d70aad7b6\" (UID: \"ef9db282-ccda-49e3-a271-934d70aad7b6\") "
Sep 11 13:50:42.758200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:42.758200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.757922    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:42.758200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758136    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:42.758380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758228    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.758380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758247    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.758380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758262    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.758380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758276    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.758380 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.758230    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:42.760053 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.760021    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:42.760187 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.760128    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:50:42.760252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.760188    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv" (OuterVolumeSpecName: "kube-api-access-b8wqv") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "kube-api-access-b8wqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:42.760252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.760190    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ef9db282-ccda-49e3-a271-934d70aad7b6" (UID: "ef9db282-ccda-49e3-a271-934d70aad7b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:50:42.858776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.858745    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8wqv\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-kube-api-access-b8wqv\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.858776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.858772    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.858776 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.858781    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.859004 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.858790    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef9db282-ccda-49e3-a271-934d70aad7b6-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:42.859004 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:42.858798    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef9db282-ccda-49e3-a271-934d70aad7b6-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:43.228006 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.227973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:43.340337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.340302    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerID="710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e" exitCode=0
Sep 11 13:50:43.340518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.340349    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"
Sep 11 13:50:43.340518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.340374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerDied","Data":"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"}
Sep 11 13:50:43.340518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.340416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n" event={"ID":"ef9db282-ccda-49e3-a271-934d70aad7b6","Type":"ContainerDied","Data":"4eebad051cfc893d27408110b443d23b70f3460b2b76d655b3025d6378300bc4"}
Sep 11 13:50:43.340518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.340437    2576 scope.go:117] "RemoveContainer" containerID="710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"
Sep 11 13:50:43.349795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.349775    2576 scope.go:117] "RemoveContainer" containerID="3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"
Sep 11 13:50:43.357329 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.357314    2576 scope.go:117] "RemoveContainer" containerID="710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"
Sep 11 13:50:43.357594 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:43.357578    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e\": container with ID starting with 710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e not found: ID does not exist" containerID="710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"
Sep 11 13:50:43.357665 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.357600    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e"} err="failed to get container status \"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e\": rpc error: code = NotFound desc = could not find container \"710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e\": container with ID starting with 710aed54eef173cbaf5465dccd0af29aad463628e3d8dcaa7fa0257546929f5e not found: ID does not exist"
Sep 11 13:50:43.357665 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.357616    2576 scope.go:117] "RemoveContainer" containerID="3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"
Sep 11 13:50:43.357826 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:43.357810    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016\": container with ID starting with 3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016 not found: ID does not exist" containerID="3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"
Sep 11 13:50:43.357871 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.357830    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016"} err="failed to get container status \"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016\": rpc error: code = NotFound desc = could not find container \"3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016\": container with ID starting with 3b1808da3f7c156f9f0f5bc58b67f29d27aaec34968a94eedf437392a1db1016 not found: ID does not exist"
Sep 11 13:50:43.367722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.367700    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:50:43.372263 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:43.372245    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-470eeb-openshift-default-846bdddbb8-skf2n"]
Sep 11 13:50:44.203448 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:44.203402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:44.248349 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:44.248307    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" path="/var/lib/kubelet/pods/ef9db282-ccda-49e3-a271-934d70aad7b6/volumes"
Sep 11 13:50:44.987356 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:44.987311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:46.408276 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:46.408247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:48.092487 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:48.092451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:48.818863 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:48.818827    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:50:48.819093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:48.819064    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" podUID="51247622-b170-463f-91e1-3511906b3563" containerName="payload-processing" containerID="cri-o://1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4" gracePeriod=30
Sep 11 13:50:48.922527 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:48.922466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:49.023545 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.023449    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:49.092323 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.092055    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:50:49.098974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.098940    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:49.224742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.224703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfdsg\" (UniqueName: \"kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg\") pod \"51247622-b170-463f-91e1-3511906b3563\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") "
Sep 11 13:50:49.224946 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.224794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume\") pod \"51247622-b170-463f-91e1-3511906b3563\" (UID: \"51247622-b170-463f-91e1-3511906b3563\") "
Sep 11 13:50:49.225205 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.225173    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51247622-b170-463f-91e1-3511906b3563" (UID: "51247622-b170-463f-91e1-3511906b3563"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:50:49.226916 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.226893    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg" (OuterVolumeSpecName: "kube-api-access-jfdsg") pod "51247622-b170-463f-91e1-3511906b3563" (UID: "51247622-b170-463f-91e1-3511906b3563"). InnerVolumeSpecName "kube-api-access-jfdsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:50:49.325720 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.325633    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfdsg\" (UniqueName: \"kubernetes.io/projected/51247622-b170-463f-91e1-3511906b3563-kube-api-access-jfdsg\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:49.325720 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.325659    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51247622-b170-463f-91e1-3511906b3563-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:50:49.372050 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.372016    2576 generic.go:356] "Generic (PLEG): container finished" podID="51247622-b170-463f-91e1-3511906b3563" containerID="1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4" exitCode=0
Sep 11 13:50:49.372212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.372059    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"
Sep 11 13:50:49.372212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.372112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" event={"ID":"51247622-b170-463f-91e1-3511906b3563","Type":"ContainerDied","Data":"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"}
Sep 11 13:50:49.372212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.372168    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p" event={"ID":"51247622-b170-463f-91e1-3511906b3563","Type":"ContainerDied","Data":"937ac5605b8f1ecae0a661d6684cabb2944dc772c5f53d027bfb10f30f7aea4f"}
Sep 11 13:50:49.372212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.372191    2576 scope.go:117] "RemoveContainer" containerID="1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"
Sep 11 13:50:49.387146 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.387127    2576 scope.go:117] "RemoveContainer" containerID="1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"
Sep 11 13:50:49.387389 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:50:49.387372    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4\": container with ID starting with 1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4 not found: ID does not exist" containerID="1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"
Sep 11 13:50:49.387430 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.387396    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4"} err="failed to get container status \"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4\": rpc error: code = NotFound desc = could not find container \"1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4\": container with ID starting with 1aaabbab46de975594c338150f66ca5c3c10afee66f1688fb693f658d06b27a4 not found: ID does not exist"
Sep 11 13:50:49.402656 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.402634    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:50:49.407130 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.407110    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-e52aaa-75fcc556d9-xsh2p"]
Sep 11 13:50:49.481890 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.481857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:49.580882 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:49.580791    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:50.248385 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:50.248353    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51247622-b170-463f-91e1-3511906b3563" path="/var/lib/kubelet/pods/51247622-b170-463f-91e1-3511906b3563/volumes"
Sep 11 13:50:50.680266 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:50.680234    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:51.580320 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:51.580291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:52.495385 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:52.495355    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.278318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.278274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.292253 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.292221    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"]
Sep 11 13:50:53.298790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.298764    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-082f62-29818905-rscnj"]
Sep 11 13:50:53.374676 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.374647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.443019 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.442988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.544906 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.544820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.746158 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.746127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:53.752020 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.751992    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"]
Sep 11 13:50:53.756847 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.756822    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-082f62-8ns6s"]
Sep 11 13:50:53.864553 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:53.864463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:54.234419 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:54.234344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:54.247829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:54.247800    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b68250e-2320-4ac5-8a33-072028530f47" path="/var/lib/kubelet/pods/0b68250e-2320-4ac5-8a33-072028530f47/volumes"
Sep 11 13:50:54.248099 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:54.248087    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa317d58-6e2e-40ef-b68e-37d76cb0749b" path="/var/lib/kubelet/pods/aa317d58-6e2e-40ef-b68e-37d76cb0749b/volumes"
Sep 11 13:50:54.813966 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:54.813939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:55.898305 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:55.898262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:56.293674 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:56.293643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:56.816069 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:56.816041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:56.914479 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:56.914447    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:57.332129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:57.332099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:50:57.332355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:57.332320    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy" containerID="cri-o://815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5" gracePeriod=30
Sep 11 13:50:57.388664 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:57.388632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:50:57.892935 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:50:57.892898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:00.823799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:00.823748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:00.888511 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:00.888466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:01.194646 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:01.194553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:02.587525 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.587480    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:51:02.751304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751222    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751261    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751304 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751282    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751316    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751369    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751391    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751415    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wq7w\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751632 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data\") pod \"8fd91f2f-9307-4e50-9f51-615418672be6\" (UID: \"8fd91f2f-9307-4e50-9f51-615418672be6\") "
Sep 11 13:51:02.751899 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751636    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:51:02.751899 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:51:02.751899 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751696    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:51:02.751899 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751779    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:51:02.752106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.751938    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data" (OuterVolumeSpecName: "istio-data") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:51:02.752106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.752015    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fd91f2f-9307-4e50-9f51-615418672be6-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.752106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.752029    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.752106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.752038    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.752106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.752046    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.753560 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.753523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:51:02.753729 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.753712    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w" (OuterVolumeSpecName: "kube-api-access-2wq7w") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "kube-api-access-2wq7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:51:02.753803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.753772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:51:02.753803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.753789    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token" (OuterVolumeSpecName: "istio-token") pod "8fd91f2f-9307-4e50-9f51-615418672be6" (UID: "8fd91f2f-9307-4e50-9f51-615418672be6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:51:02.852478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.852444    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fd91f2f-9307-4e50-9f51-615418672be6-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.852478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.852473    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.852478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.852482    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wq7w\" (UniqueName: \"kubernetes.io/projected/8fd91f2f-9307-4e50-9f51-615418672be6-kube-api-access-2wq7w\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.852702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.852491    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:02.852702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:02.852522    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fd91f2f-9307-4e50-9f51-615418672be6-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:03.444204 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.444171    2576 generic.go:356] "Generic (PLEG): container finished" podID="8fd91f2f-9307-4e50-9f51-615418672be6" containerID="815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5" exitCode=0
Sep 11 13:51:03.444382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.444220    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"
Sep 11 13:51:03.444382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.444251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerDied","Data":"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"}
Sep 11 13:51:03.444382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.444307    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg" event={"ID":"8fd91f2f-9307-4e50-9f51-615418672be6","Type":"ContainerDied","Data":"2df4ec01a6b9bc0541e112ca7523f5d411da4d795bffc4f7c0fc1117c04d43b2"}
Sep 11 13:51:03.444382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.444329    2576 scope.go:117] "RemoveContainer" containerID="815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"
Sep 11 13:51:03.454017 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.453890    2576 scope.go:117] "RemoveContainer" containerID="1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"
Sep 11 13:51:03.462103 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.462085    2576 scope.go:117] "RemoveContainer" containerID="815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"
Sep 11 13:51:03.462371 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:51:03.462353    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5\": container with ID starting with 815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5 not found: ID does not exist" containerID="815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"
Sep 11 13:51:03.462440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.462378    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5"} err="failed to get container status \"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5\": rpc error: code = NotFound desc = could not find container \"815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5\": container with ID starting with 815107b7640091d62e97d9b1fc3287a6de30fa922d480fcf4b7e9eed8a0505e5 not found: ID does not exist"
Sep 11 13:51:03.462440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.462392    2576 scope.go:117] "RemoveContainer" containerID="1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"
Sep 11 13:51:03.462654 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:51:03.462640    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4\": container with ID starting with 1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4 not found: ID does not exist" containerID="1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"
Sep 11 13:51:03.462697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.462658    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4"} err="failed to get container status \"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4\": rpc error: code = NotFound desc = could not find container \"1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4\": container with ID starting with 1ff6f818d423b833a138b8afd15fe604aea9473a41abd6593d977160684259a4 not found: ID does not exist"
Sep 11 13:51:03.472017 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.471993    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:51:03.476817 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:03.476793    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-e52aaa-openshift-default-57894659fd-qmrsg"]
Sep 11 13:51:04.248335 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:04.248302    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" path="/var/lib/kubelet/pods/8fd91f2f-9307-4e50-9f51-615418672be6/volumes"
Sep 11 13:51:16.352722 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:16.352684    2576 scope.go:117] "RemoveContainer" containerID="e831a2625f90372fcdb52e3e3e07bee7e3d175f1aa100c151a2e0b7cb5b1e8a2"
Sep 11 13:51:16.360715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:16.360694    2576 scope.go:117] "RemoveContainer" containerID="99e818c85d3c0228e55f4dff32506eac95c7b7be4da990a28152453d8cb745fd"
Sep 11 13:51:17.511912 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:17.511871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:17.613534 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:17.613487    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:19.267988 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:19.267955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:34.506440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:34.506386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:54.386455 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.386428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:54.526438 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526406    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"]
Sep 11 13:51:54.526735 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526723    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b68250e-2320-4ac5-8a33-072028530f47" containerName="revoke-keys"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526737    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b68250e-2320-4ac5-8a33-072028530f47" containerName="revoke-keys"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526746    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526752    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526758    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51247622-b170-463f-91e1-3511906b3563" containerName="payload-processing"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526764    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="51247622-b170-463f-91e1-3511906b3563" containerName="payload-processing"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526770    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.526780 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526775    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526790    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526796    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526846    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526855    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526862    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="51247622-b170-463f-91e1-3511906b3563" containerName="payload-processing"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526867    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b68250e-2320-4ac5-8a33-072028530f47" containerName="revoke-keys"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526872    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.526989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.526878    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fd91f2f-9307-4e50-9f51-615418672be6" containerName="istio-proxy"
Sep 11 13:51:54.529707 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.529691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:54.532264 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.532243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:51:54.532387 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.532247    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:51:54.540028 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.540007    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"]
Sep 11 13:51:54.608706 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.608680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9\" (UID: \"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:54.709447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.709380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9\" (UID: \"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:54.709964 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.709947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9\" (UID: \"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:54.840151 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:54.840124    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:55.110844 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:55.110813    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"]
Sep 11 13:51:55.112667 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:51:55.112629    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff23d8a_1c3d_4ed3_95f4_8c81ccdfc624.slice/crio-443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803 WatchSource:0}: Error finding container 443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803: Status 404 returned error can't find the container with id 443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803
Sep 11 13:51:55.693185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:55.693147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9" event={"ID":"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624","Type":"ContainerStarted","Data":"b2ba672f2dfffebb87904a8406f37c4b123b77297c0d674b327e6240624f9c6f"}
Sep 11 13:51:55.693593 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:55.693199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9" event={"ID":"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624","Type":"ContainerStarted","Data":"443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803"}
Sep 11 13:51:55.719752 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:55.719709    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9" podStartSLOduration=1.719693871 podStartE2EDuration="1.719693871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:51:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:51:55.719077732 +0000 UTC m=+1840.112765153" watchObservedRunningTime="2026-09-11 13:51:55.719693871 +0000 UTC m=+1840.113381281"
Sep 11 13:51:55.784384 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:55.784357    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:56.699941 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:56.699905    2576 generic.go:356] "Generic (PLEG): container finished" podID="bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" containerID="b2ba672f2dfffebb87904a8406f37c4b123b77297c0d674b327e6240624f9c6f" exitCode=0
Sep 11 13:51:56.700315 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:56.699988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9" event={"ID":"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624","Type":"ContainerDied","Data":"b2ba672f2dfffebb87904a8406f37c4b123b77297c0d674b327e6240624f9c6f"}
Sep 11 13:51:56.830185 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:56.830151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:57.717565 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:57.717538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:57.844011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:57.843988    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:57.936866 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:57.936843    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca\") pod \"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624\" (UID: \"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624\") "
Sep 11 13:51:57.937170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:57.937148    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" (UID: "bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:51:58.038124 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:58.038103    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:51:58.689848 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:58.689819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:51:58.711478 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:58.711443    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"
Sep 11 13:51:58.711669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:58.711473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9" event={"ID":"bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624","Type":"ContainerDied","Data":"443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803"}
Sep 11 13:51:58.711669 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:58.711550    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="443f8458fd996ce9b1491aac9b34099ca083ded0c8a854ee0042dd953a8fd803"
Sep 11 13:51:59.365231 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:59.365194    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:51:59.445002 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:51:59.444970    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:10.437677 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.437625    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:10.487075 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.487042    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"]
Sep 11 13:52:10.491809 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.491787    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-371c38-29818905-5kn4n"]
Sep 11 13:52:10.614616 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.614580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:10.713587 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.713511    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:10.953885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:10.953837    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:11.063922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:11.063875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:11.070600 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:11.070569    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"]
Sep 11 13:52:11.077842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:11.077806    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-371c38-drrl9"]
Sep 11 13:52:11.451684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:11.451600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:11.966150 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:11.966122    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:12.248365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:12.248271    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" path="/var/lib/kubelet/pods/bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624/volumes"
Sep 11 13:52:12.248601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:12.248585    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e536965e-bf3e-4f23-8429-09a71d878b3c" path="/var/lib/kubelet/pods/e536965e-bf3e-4f23-8429-09a71d878b3c/volumes"
Sep 11 13:52:13.217393 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:13.217364    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:14.372959 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:14.372927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:14.736125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:14.736045    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" podUID="42292481-a10b-4d13-8afe-8056569d3cc5" containerName="main" containerID="cri-o://e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0" gracePeriod=45
Sep 11 13:52:14.984628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:14.984603    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:52:15.084194 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084166    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084235    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5g67\" (UniqueName: \"kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084263    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084282    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084383    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084456    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home\") pod \"42292481-a10b-4d13-8afe-8056569d3cc5\" (UID: \"42292481-a10b-4d13-8afe-8056569d3cc5\") "
Sep 11 13:52:15.084582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084470    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:15.084582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084567    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache" (OuterVolumeSpecName: "model-cache") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:15.084795 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084781    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-model-cache\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.084843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084803    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-tmp-dir\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.084843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.084776    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home" (OuterVolumeSpecName: "home") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:15.086474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.086439    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm" (OuterVolumeSpecName: "dshm") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:15.086623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.086531    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67" (OuterVolumeSpecName: "kube-api-access-r5g67") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "kube-api-access-r5g67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:15.086623 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.086553    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "42292481-a10b-4d13-8afe-8056569d3cc5" (UID: "42292481-a10b-4d13-8afe-8056569d3cc5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:52:15.185424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.185391    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5g67\" (UniqueName: \"kubernetes.io/projected/42292481-a10b-4d13-8afe-8056569d3cc5-kube-api-access-r5g67\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.185424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.185420    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/42292481-a10b-4d13-8afe-8056569d3cc5-tls-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.185424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.185430    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-dshm\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.185675 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.185438    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/42292481-a10b-4d13-8afe-8056569d3cc5-home\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:15.792882 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.792849    2576 generic.go:356] "Generic (PLEG): container finished" podID="42292481-a10b-4d13-8afe-8056569d3cc5" containerID="e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0" exitCode=0
Sep 11 13:52:15.793282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.792889    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"
Sep 11 13:52:15.793282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.792918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" event={"ID":"42292481-a10b-4d13-8afe-8056569d3cc5","Type":"ContainerDied","Data":"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"}
Sep 11 13:52:15.793282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.792947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb" event={"ID":"42292481-a10b-4d13-8afe-8056569d3cc5","Type":"ContainerDied","Data":"cb905a92753729459b3b09ac00093fa401c3b53cb62feafe01e149474965cbc1"}
Sep 11 13:52:15.793282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.792962    2576 scope.go:117] "RemoveContainer" containerID="e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"
Sep 11 13:52:15.801394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.801379    2576 scope.go:117] "RemoveContainer" containerID="e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"
Sep 11 13:52:15.801665 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:52:15.801645    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0\": container with ID starting with e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0 not found: ID does not exist" containerID="e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"
Sep 11 13:52:15.801743 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.801675    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0"} err="failed to get container status \"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0\": rpc error: code = NotFound desc = could not find container \"e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0\": container with ID starting with e26a907e26864c691c16f04af2d2780c213178cb4903dcf546ff60e607d586e0 not found: ID does not exist"
Sep 11 13:52:15.831331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.831309    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:52:15.839458 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:15.839435    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-371c38/distinct-2-w4-371c38-kserve-68b9b44d5-nsrtb"]
Sep 11 13:52:16.248508 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:16.248416    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42292481-a10b-4d13-8afe-8056569d3cc5" path="/var/lib/kubelet/pods/42292481-a10b-4d13-8afe-8056569d3cc5/volumes"
Sep 11 13:52:16.398087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:16.398058    2576 scope.go:117] "RemoveContainer" containerID="e47796093bee3eef933fc7977401c6b00cf8dd5488ae7ef61dcd098e6fb53398"
Sep 11 13:52:16.405551 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:16.405528    2576 scope.go:117] "RemoveContainer" containerID="ffb87df00a569f409d4bac337c1338d847a8a5a639a6ef216f8ec04af180be3f"
Sep 11 13:52:17.906769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:17.906731    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:52:17.926236 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:17.926209    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:18.985322 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:18.985296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:19.053367 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:19.053338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:19.287628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:19.287600    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:20.753979 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:20.753949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:28.559345 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:28.559314    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:29.252760 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:29.252727    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:32.924909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:32.924874    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" podUID="688e62fb-00f5-4839-983d-1a1b8f123d27" containerName="main" containerID="cri-o://cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758" gracePeriod=45
Sep 11 13:52:33.176859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.176795    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:52:33.351794 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.351761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.351968 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.351826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.351968 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.351872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.351968 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.351938    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt2zk\" (UniqueName: \"kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.352160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.351963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.352160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:33.352160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs\") pod \"688e62fb-00f5-4839-983d-1a1b8f123d27\" (UID: \"688e62fb-00f5-4839-983d-1a1b8f123d27\") "
Sep 11 13:52:33.352160 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352110    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache" (OuterVolumeSpecName: "model-cache") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:33.352363 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home" (OuterVolumeSpecName: "home") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:33.352418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352386    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-tmp-dir\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.352418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352404    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-model-cache\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.352491 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.352417    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-home\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.354256 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.354236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm" (OuterVolumeSpecName: "dshm") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:52:33.354256 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.354241    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:52:33.354379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.354330    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk" (OuterVolumeSpecName: "kube-api-access-dt2zk") pod "688e62fb-00f5-4839-983d-1a1b8f123d27" (UID: "688e62fb-00f5-4839-983d-1a1b8f123d27"). InnerVolumeSpecName "kube-api-access-dt2zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:52:33.453306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.453238    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688e62fb-00f5-4839-983d-1a1b8f123d27-dshm\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.453306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.453267    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt2zk\" (UniqueName: \"kubernetes.io/projected/688e62fb-00f5-4839-983d-1a1b8f123d27-kube-api-access-dt2zk\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.453306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.453279    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688e62fb-00f5-4839-983d-1a1b8f123d27-tls-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:52:33.880732 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.880699    2576 generic.go:356] "Generic (PLEG): container finished" podID="688e62fb-00f5-4839-983d-1a1b8f123d27" containerID="cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758" exitCode=0
Sep 11 13:52:33.880908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.880742    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"
Sep 11 13:52:33.880908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.880774    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" event={"ID":"688e62fb-00f5-4839-983d-1a1b8f123d27","Type":"ContainerDied","Data":"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"}
Sep 11 13:52:33.880908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.880817    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp" event={"ID":"688e62fb-00f5-4839-983d-1a1b8f123d27","Type":"ContainerDied","Data":"7c50c28d640b5173f3b25ff60c8be996e9b9cac743b0a7858fda17d876b30f07"}
Sep 11 13:52:33.880908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.880834    2576 scope.go:117] "RemoveContainer" containerID="cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"
Sep 11 13:52:33.889424 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.889408    2576 scope.go:117] "RemoveContainer" containerID="cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"
Sep 11 13:52:33.889736 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:52:33.889719    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758\": container with ID starting with cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758 not found: ID does not exist" containerID="cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"
Sep 11 13:52:33.889803 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.889742    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758"} err="failed to get container status \"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758\": rpc error: code = NotFound desc = could not find container \"cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758\": container with ID starting with cd67c6d68f79c5c88e7b9baba24a4e45635209b7efb03939fa3e734c19a55758 not found: ID does not exist"
Sep 11 13:52:33.933238 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.933211    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:52:33.943205 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:33.943183    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-0317c3/auth-test-model-0317c3-kserve-6d7f774c99-tbdfp"]
Sep 11 13:52:34.248742 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:34.248664    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="688e62fb-00f5-4839-983d-1a1b8f123d27" path="/var/lib/kubelet/pods/688e62fb-00f5-4839-983d-1a1b8f123d27/volumes"
Sep 11 13:52:37.137999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:37.137964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:37.735865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:37.735822    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:37.809452 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:37.809423    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:38.458517 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:38.458421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:45.377234 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:45.377200    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:52:53.465888 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:52:53.465851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:53:34.998312 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:53:34.998280    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:53:36.505704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:53:36.505670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:53:48.434202 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:53:48.434171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:53:52.686891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:53:52.686864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:54:30.543225 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:54:30.543191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:54:43.292662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:54:43.292626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:54:45.598902 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:54:45.598867    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:11.284524 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:11.284474    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:12.468868 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:12.468827    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:12.921426 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:12.921391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:13.277925 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:13.277892    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:13.429201 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:13.429168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:16.103936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:16.103904    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:16.721224 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:16.721185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:16.821704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:16.821673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:17.184451 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:17.184424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:18.631856 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:18.631824    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:19.518303 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:19.518270    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:19.602773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:19.602738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:19.703080 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:19.703045    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:20.161328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.161294    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:20.333461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.333424    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:20.424414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424332    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:55:20.424715 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424699    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="688e62fb-00f5-4839-983d-1a1b8f123d27" containerName="main"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424717    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="688e62fb-00f5-4839-983d-1a1b8f123d27" containerName="main"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424729    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42292481-a10b-4d13-8afe-8056569d3cc5" containerName="main"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424734    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42292481-a10b-4d13-8afe-8056569d3cc5" containerName="main"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424748    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424754    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef9db282-ccda-49e3-a271-934d70aad7b6" containerName="istio-proxy"
Sep 11 13:55:20.424764 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424762    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" containerName="revoke-keys"
Sep 11 13:55:20.424965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424767    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" containerName="revoke-keys"
Sep 11 13:55:20.424965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424818    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="688e62fb-00f5-4839-983d-1a1b8f123d27" containerName="main"
Sep 11 13:55:20.424965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424824    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42292481-a10b-4d13-8afe-8056569d3cc5" containerName="main"
Sep 11 13:55:20.424965 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.424830    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bff23d8a-1c3d-4ed3-95f4-8c81ccdfc624" containerName="revoke-keys"
Sep 11 13:55:20.427782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.427766    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.430829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.430806    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-264b08-dockercfg-zj99k\""
Sep 11 13:55:20.430919 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.430806    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-264b08\""
Sep 11 13:55:20.444887 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.444862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:55:20.474136 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.474102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.474328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.474144    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgq2s\" (UniqueName: \"kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.575561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.575515    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.575561 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.575563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgq2s\" (UniqueName: \"kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.576222 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.576203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.586606 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.586574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgq2s\" (UniqueName: \"kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s\") pod \"payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.737883 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.737806    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:20.771383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:20.771344    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:21.008353 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.008329    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:55:21.010426 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:55:21.010391    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719bb040_6e73_4ffe_b4bf_5e9f93f26796.slice/crio-c45d14cabe77d6c093db5c426e5beee29fbf87f51ba732d385d48e11c4f7c10b WatchSource:0}: Error finding container c45d14cabe77d6c093db5c426e5beee29fbf87f51ba732d385d48e11c4f7c10b: Status 404 returned error can't find the container with id c45d14cabe77d6c093db5c426e5beee29fbf87f51ba732d385d48e11c4f7c10b
Sep 11 13:55:21.644682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.644648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" event={"ID":"719bb040-6e73-4ffe-b4bf-5e9f93f26796","Type":"ContainerStarted","Data":"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"}
Sep 11 13:55:21.644879 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.644700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" event={"ID":"719bb040-6e73-4ffe-b4bf-5e9f93f26796","Type":"ContainerStarted","Data":"c45d14cabe77d6c093db5c426e5beee29fbf87f51ba732d385d48e11c4f7c10b"}
Sep 11 13:55:21.644879 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.644847    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:21.663779 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.663724    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" podStartSLOduration=1.66370889 podStartE2EDuration="1.66370889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:21.661636737 +0000 UTC m=+2046.055324148" watchObservedRunningTime="2026-09-11 13:55:21.66370889 +0000 UTC m=+2046.057396301"
Sep 11 13:55:21.711141 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:21.711107    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:26.546606 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:26.546577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:26.613802 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:26.613770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:32.652118 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:32.652088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:55:41.792107 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:41.792018    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:42.233474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.233352    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:42.519990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.519960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:42.812647 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.812558    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:55:42.816123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.816103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.818383 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.818365    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-a403bc-openshift-default-dockercfg-ns5h8\""
Sep 11 13:55:42.831252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.831230    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:55:42.841926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.841902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:42.878423 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878728 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndqk9\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.878839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.878731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.942595 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.942560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:42.979186 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979154    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndqk9\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979773 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.979983 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.979827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.980302 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.980281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.981762 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.981741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.981851 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.981828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.988771 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.988737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndqk9\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:42.992182 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:42.992157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token\") pod \"e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") " pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:43.126081 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.125990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:43.295951 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.295916    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:43.399526 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.399423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:55:43.402398 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:55:43.402365    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b2e53bc_f8cd_444c_bf15_f9bc4f5689ed.slice/crio-fd414f23f9eadb13b6d3e2a022cdfb8bf280db2860eb8f902fa1bc8266c93533 WatchSource:0}: Error finding container fd414f23f9eadb13b6d3e2a022cdfb8bf280db2860eb8f902fa1bc8266c93533: Status 404 returned error can't find the container with id fd414f23f9eadb13b6d3e2a022cdfb8bf280db2860eb8f902fa1bc8266c93533
Sep 11 13:55:43.404580 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.404546    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:55:43.404655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.404613    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:55:43.404655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.404645    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:55:43.747778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.747678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerStarted","Data":"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"}
Sep 11 13:55:43.747778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.747749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerStarted","Data":"fd414f23f9eadb13b6d3e2a022cdfb8bf280db2860eb8f902fa1bc8266c93533"}
Sep 11 13:55:43.774821 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:43.774763    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podStartSLOduration=1.77474622 podStartE2EDuration="1.77474622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:55:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:55:43.773308626 +0000 UTC m=+2068.166996039" watchObservedRunningTime="2026-09-11 13:55:43.77474622 +0000 UTC m=+2068.168433632"
Sep 11 13:55:44.126198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:44.126158    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:44.131170 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:44.131144    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:44.300212 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:44.300179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:44.757709 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:44.757663    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:44.758651 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:44.758630    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:55:45.287112 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:45.287077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:45.829465 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:45.829438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:45.942268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:45.942237    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:46.418721 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:46.418691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:47.399328 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:47.399291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:48.671706 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:48.671668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:49.475257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:49.475197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:49.575760 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:49.575733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:49.640814 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:49.640787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:49.979748 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:49.979718    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:50.053135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:50.053102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:58.584289 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:58.584255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:55:58.651060 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:55:58.651010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:05.863404 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.863369    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerID="5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45" exitCode=0
Sep 11 13:56:05.863841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.863412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerDied","Data":"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"}
Sep 11 13:56:05.863841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.863820    2576 scope.go:117] "RemoveContainer" containerID="5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"
Sep 11 13:56:05.864947 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.864913    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:05.865012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.864983    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:05.865057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:05.865020    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:06.869892 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:06.869854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerStarted","Data":"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"}
Sep 11 13:56:07.127481 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:07.127383    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:56:07.127845 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:07.127820    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 11 13:56:07.128016 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:07.127991    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 11 13:56:08.127337 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:08.127286    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 11 13:56:08.127858 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:08.127359    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 11 13:56:09.126636 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:09.126593    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 11 13:56:09.126979 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:09.126943    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 11 13:56:10.127909 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:10.127871    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 11 13:56:10.128528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:10.128479    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 11 13:56:11.126701 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:11.126664    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 11 13:56:11.127015 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:11.126986    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 11 13:56:11.883997 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:11.883965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:12.130402 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:12.130373    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:56:12.130754 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:12.130730    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:56:12.131327 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:12.131310    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:56:12.780223 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:12.780185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:13.792790 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:13.792756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:14.587573 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:14.587537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:15.479265 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:15.479230    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:15.964386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:15.964358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:25.557063 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:25.557021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:57.907842 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:57.907810    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:58.317659 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.317622    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:56:58.321054 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.321027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.323639 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.323616    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-7b7819-openshift-default-dockercfg-drs4k\""
Sep 11 13:56:58.327270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.327248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:58.334409 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.334386    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:56:58.372829 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm9zl\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.372990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.372990    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.373225 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.373022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.373225 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.373049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.473810 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.473771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.473989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.473855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.473989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.473883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.473989 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.473925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm9zl\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474159 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474167    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474220 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474195    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474295    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474326 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474233    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.474551 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.474454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.475116 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.475091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.475414 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.475388    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.476474 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.476443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.477106 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.477088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.482588 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.482519    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm9zl\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.483471 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.483447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token\") pod \"e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") " pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.600057 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.599967    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:56:58.635045 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.635007    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:58.916070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.915992    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:56:58.919388 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:56:58.919359    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f2e5ffd_ecc6_4986_818c_4b705708411a.slice/crio-82fe2585c87e70b86ae2b72878d1fc225693c76959f5482c88afbb0a0584ca15 WatchSource:0}: Error finding container 82fe2585c87e70b86ae2b72878d1fc225693c76959f5482c88afbb0a0584ca15: Status 404 returned error can't find the container with id 82fe2585c87e70b86ae2b72878d1fc225693c76959f5482c88afbb0a0584ca15
Sep 11 13:56:58.922135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.922102    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:58.922215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.922173    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:58.922215 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:58.922203    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:56:59.116368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:59.116319    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerStarted","Data":"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"}
Sep 11 13:56:59.116555 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:59.116409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerStarted","Data":"82fe2585c87e70b86ae2b72878d1fc225693c76959f5482c88afbb0a0584ca15"}
Sep 11 13:56:59.143833 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:59.143784    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podStartSLOduration=1.143766587 podStartE2EDuration="1.143766587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:56:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:56:59.142341093 +0000 UTC m=+2143.536028505" watchObservedRunningTime="2026-09-11 13:56:59.143766587 +0000 UTC m=+2143.537454003"
Sep 11 13:56:59.635425 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:59.635396    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:56:59.640738 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:56:59.640712    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:00.120831 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:00.120799    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:00.121798 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:00.121778    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:00.557300 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:00.557267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:01.128814 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:01.128778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:01.227900 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:01.227871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:01.811322 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:01.811292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:02.899475 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:02.899445    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:03.709640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:03.709608    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:04.620610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:04.620575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:05.630944 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:05.630909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:05.693931 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:05.693901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:05.794950 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:05.794917    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:06.234955 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.234924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:06.294376 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.294347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:06.389627 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.389589    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:06.392996 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.392976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.395865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.395844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-7b7819-dockercfg-2xsrm\""
Sep 11 13:57:06.395964 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.395847    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-7b7819\""
Sep 11 13:57:06.407003 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.406981    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:06.453292 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.453248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.453679 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.453651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgz4q\" (UniqueName: \"kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.554769 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.554738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.554954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.554813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgz4q\" (UniqueName: \"kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.555461 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.555440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.564960 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.564935    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgz4q\" (UniqueName: \"kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q\") pod \"payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.703072 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.703038    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:06.980193 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:06.980169    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:06.982138 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:57:06.982109    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f221115_4269_40db_b1fc_8fde16cfffb4.slice/crio-7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba WatchSource:0}: Error finding container 7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba: Status 404 returned error can't find the container with id 7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba
Sep 11 13:57:07.155859 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:07.155828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" event={"ID":"5f221115-4269-40db-b1fc-8fde16cfffb4","Type":"ContainerStarted","Data":"cfea7e18ecd8906cbb92d27f7c341ad569156ee6174a27c7d1706d6eba6c9669"}
Sep 11 13:57:07.156046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:07.155880    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" event={"ID":"5f221115-4269-40db-b1fc-8fde16cfffb4","Type":"ContainerStarted","Data":"7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba"}
Sep 11 13:57:07.156046 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:07.155910    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:07.197959 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:07.197906    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" podStartSLOduration=1.1978897370000001 podStartE2EDuration="1.197889737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:07.196733818 +0000 UTC m=+2151.590421230" watchObservedRunningTime="2026-09-11 13:57:07.197889737 +0000 UTC m=+2151.591577147"
Sep 11 13:57:16.461247 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:16.461170    2576 scope.go:117] "RemoveContainer" containerID="3444292794fd85b21df84166fa40a4d369727baa705b99a481874b61b0549968"
Sep 11 13:57:16.469512 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:16.469481    2576 scope.go:117] "RemoveContainer" containerID="2f523f46ad22afdac342ae51bf99e60aa2b65809da344135b66695a579b21022"
Sep 11 13:57:18.162473 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:18.162441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:22.249373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.249299    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerID="cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce" exitCode=0
Sep 11 13:57:22.249373 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.249368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerDied","Data":"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"}
Sep 11 13:57:22.249813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.249763    2576 scope.go:117] "RemoveContainer" containerID="cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"
Sep 11 13:57:22.250880 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.250852    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:57:22.250953 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.250925    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:57:22.250994 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:22.250956    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:57:23.256203 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:23.256163    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerStarted","Data":"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"}
Sep 11 13:57:23.635230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:23.635189    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:23.635440 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:23.635415    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 11 13:57:23.635486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:23.635468    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 11 13:57:24.635704 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:24.635666    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 11 13:57:24.636124 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:24.635723    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 11 13:57:25.635682 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:25.635639    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 11 13:57:25.635936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:25.635710    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 11 13:57:26.636270 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:26.636229    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 11 13:57:26.636784 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:26.636294    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 11 13:57:27.635610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:27.635571    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 11 13:57:27.635798 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:27.635639    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 11 13:57:27.986729 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:27.986635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:28.705886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:28.705836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:28.706364 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:28.706339    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:28.707557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:28.707537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:28.793541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:28.793483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:29.586567 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:29.586540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:30.508684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:30.508652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:31.356599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:31.356567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:31.456911 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:31.456880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:32.182431 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.182398    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"]
Sep 11 13:57:32.191483 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.191454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:32.193466 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.193435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"]
Sep 11 13:57:32.193959 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.193935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:57:32.194087 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.193941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:57:32.298832 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.298796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x\" (UID: \"6c9757e2-42a1-4feb-a224-897b974ebdfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:32.400125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.400087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x\" (UID: \"6c9757e2-42a1-4feb-a224-897b974ebdfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:32.400889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.400863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x\" (UID: \"6c9757e2-42a1-4feb-a224-897b974ebdfc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:32.502910 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.502825    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:32.608287 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.608256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:32.773198 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:32.773173    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"]
Sep 11 13:57:32.775045 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:57:32.775019    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9757e2_42a1_4feb_a224_897b974ebdfc.slice/crio-8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55 WatchSource:0}: Error finding container 8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55: Status 404 returned error can't find the container with id 8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55
Sep 11 13:57:33.302093 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:33.302059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x" event={"ID":"6c9757e2-42a1-4feb-a224-897b974ebdfc","Type":"ContainerStarted","Data":"f38106ec293b46e5eb2c646fd4d20f3148aede6a5dd1f5af5c5ae5181eeef0ff"}
Sep 11 13:57:33.302275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:33.302112    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x" event={"ID":"6c9757e2-42a1-4feb-a224-897b974ebdfc","Type":"ContainerStarted","Data":"8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55"}
Sep 11 13:57:33.320078 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:33.320027    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x" podStartSLOduration=1.320011344 podStartE2EDuration="1.320011344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:33.318255858 +0000 UTC m=+2177.711943269" watchObservedRunningTime="2026-09-11 13:57:33.320011344 +0000 UTC m=+2177.713698755"
Sep 11 13:57:33.437952 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:33.437909    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:34.308718 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:34.308684    2576 generic.go:356] "Generic (PLEG): container finished" podID="6c9757e2-42a1-4feb-a224-897b974ebdfc" containerID="f38106ec293b46e5eb2c646fd4d20f3148aede6a5dd1f5af5c5ae5181eeef0ff" exitCode=0
Sep 11 13:57:34.308907 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:34.308774    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x" event={"ID":"6c9757e2-42a1-4feb-a224-897b974ebdfc","Type":"ContainerDied","Data":"f38106ec293b46e5eb2c646fd4d20f3148aede6a5dd1f5af5c5ae5181eeef0ff"}
Sep 11 13:57:34.404688 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:34.404657    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:35.173540 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:35.173491    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:35.442767 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:35.442743    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:35.529746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:35.529712    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca\") pod \"6c9757e2-42a1-4feb-a224-897b974ebdfc\" (UID: \"6c9757e2-42a1-4feb-a224-897b974ebdfc\") "
Sep 11 13:57:35.530070 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:35.530048    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6c9757e2-42a1-4feb-a224-897b974ebdfc" (UID: "6c9757e2-42a1-4feb-a224-897b974ebdfc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:57:35.631397 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:35.631363    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6c9757e2-42a1-4feb-a224-897b974ebdfc-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:36.147418 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:36.147382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:36.319339 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:36.319303    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x" event={"ID":"6c9757e2-42a1-4feb-a224-897b974ebdfc","Type":"ContainerDied","Data":"8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55"}
Sep 11 13:57:36.319339 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:36.319331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"
Sep 11 13:57:36.319813 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:36.319356    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f408e911b2c61bc2dd291697a56c7dcc892287e2f63dfa558f1133c984ecc55"
Sep 11 13:57:36.912882 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:36.912850    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:37.740656 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:37.740626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:38.684708 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:38.684677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.039957 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.039928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.101608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.101575    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:43.101839 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.101816    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" podUID="5f221115-4269-40db-b1fc-8fde16cfffb4" containerName="payload-pre-processing" containerID="cri-o://cfea7e18ecd8906cbb92d27f7c341ad569156ee6174a27c7d1706d6eba6c9669" gracePeriod=30
Sep 11 13:57:43.124782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.124753    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.186296 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.186261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.291225 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.291010    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"]
Sep 11 13:57:43.291697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.291673    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.293559 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.293486    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-7b7819-wdm2x"]
Sep 11 13:57:43.352974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.352944    2576 generic.go:356] "Generic (PLEG): container finished" podID="5f221115-4269-40db-b1fc-8fde16cfffb4" containerID="cfea7e18ecd8906cbb92d27f7c341ad569156ee6174a27c7d1706d6eba6c9669" exitCode=0
Sep 11 13:57:43.353122 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.353008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" event={"ID":"5f221115-4269-40db-b1fc-8fde16cfffb4","Type":"ContainerDied","Data":"cfea7e18ecd8906cbb92d27f7c341ad569156ee6174a27c7d1706d6eba6c9669"}
Sep 11 13:57:43.353122 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.353065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75" event={"ID":"5f221115-4269-40db-b1fc-8fde16cfffb4","Type":"ContainerDied","Data":"7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba"}
Sep 11 13:57:43.353122 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.353082    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f92929ee3cbeea2090794573f0ad6c82dbd8be42104d3373d8b0070b11da1ba"
Sep 11 13:57:43.354868 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.354852    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:43.376596 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.376563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.412741 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.412703    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgz4q\" (UniqueName: \"kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q\") pod \"5f221115-4269-40db-b1fc-8fde16cfffb4\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") "
Sep 11 13:57:43.412886 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.412788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume\") pod \"5f221115-4269-40db-b1fc-8fde16cfffb4\" (UID: \"5f221115-4269-40db-b1fc-8fde16cfffb4\") "
Sep 11 13:57:43.413133 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.413110    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f221115-4269-40db-b1fc-8fde16cfffb4" (UID: "5f221115-4269-40db-b1fc-8fde16cfffb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:57:43.414998 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.414977    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q" (OuterVolumeSpecName: "kube-api-access-hgz4q") pod "5f221115-4269-40db-b1fc-8fde16cfffb4" (UID: "5f221115-4269-40db-b1fc-8fde16cfffb4"). InnerVolumeSpecName "kube-api-access-hgz4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:57:43.514282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.514246    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgz4q\" (UniqueName: \"kubernetes.io/projected/5f221115-4269-40db-b1fc-8fde16cfffb4-kube-api-access-hgz4q\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:43.514282 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.514274    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f221115-4269-40db-b1fc-8fde16cfffb4-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:43.677259 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.677180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:43.872566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:43.872532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:44.248638 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:44.248601    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c9757e2-42a1-4feb-a224-897b974ebdfc" path="/var/lib/kubelet/pods/6c9757e2-42a1-4feb-a224-897b974ebdfc/volumes"
Sep 11 13:57:44.362365 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:44.362314    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"
Sep 11 13:57:44.380811 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:44.380778    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:44.384889 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:44.384863    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7b7819-f76474598-cbd75"]
Sep 11 13:57:44.617331 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:44.617295    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:45.732782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:45.732752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:46.247644 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:46.247612    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f221115-4269-40db-b1fc-8fde16cfffb4" path="/var/lib/kubelet/pods/5f221115-4269-40db-b1fc-8fde16cfffb4/volumes"
Sep 11 13:57:46.554230 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:46.554202    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:47.596244 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:47.596212    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:50.149001 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:50.148965    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:50.218833 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:50.218800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:50.437460 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:50.437384    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:57:50.437640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:50.437607    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy" containerID="cri-o://868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb" gracePeriod=30
Sep 11 13:57:50.452634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:50.452609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:52.479370 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:52.479337    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:54.307013 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.306986    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"]
Sep 11 13:57:54.307388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307305    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f221115-4269-40db-b1fc-8fde16cfffb4" containerName="payload-pre-processing"
Sep 11 13:57:54.307388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307317    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f221115-4269-40db-b1fc-8fde16cfffb4" containerName="payload-pre-processing"
Sep 11 13:57:54.307388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307338    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c9757e2-42a1-4feb-a224-897b974ebdfc" containerName="revoke-keys"
Sep 11 13:57:54.307388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307344    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c9757e2-42a1-4feb-a224-897b974ebdfc" containerName="revoke-keys"
Sep 11 13:57:54.307530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307395    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c9757e2-42a1-4feb-a224-897b974ebdfc" containerName="revoke-keys"
Sep 11 13:57:54.307530 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.307405    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f221115-4269-40db-b1fc-8fde16cfffb4" containerName="payload-pre-processing"
Sep 11 13:57:54.309798 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.309779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:54.311891 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.311868    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:57:54.311992 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.311924    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 13:57:54.317881 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.317857    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"]
Sep 11 13:57:54.414379 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.414349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8\" (UID: \"dca04d05-1c08-498d-9889-8a1b9c5007e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:54.515275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.515228    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8\" (UID: \"dca04d05-1c08-498d-9889-8a1b9c5007e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:54.516165 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.516128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8\" (UID: \"dca04d05-1c08-498d-9889-8a1b9c5007e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:54.520268 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.520238    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:54.620657 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.620559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:54.897275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:54.897245    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"]
Sep 11 13:57:54.899725 ip-10-0-133-221 kubenswrapper[2576]: W0911 13:57:54.899699    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca04d05_1c08_498d_9889_8a1b9c5007e2.slice/crio-0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0 WatchSource:0}: Error finding container 0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0: Status 404 returned error can't find the container with id 0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0
Sep 11 13:57:55.346777 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.346745    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:55.424257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.424213    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8" event={"ID":"dca04d05-1c08-498d-9889-8a1b9c5007e2","Type":"ContainerStarted","Data":"1f9002238031c58bf9f9d8c91f1dcef560358a04798e4b77075895bbe39ae185"}
Sep 11 13:57:55.424462 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.424304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8" event={"ID":"dca04d05-1c08-498d-9889-8a1b9c5007e2","Type":"ContainerStarted","Data":"0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0"}
Sep 11 13:57:55.444557 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.444490    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8" podStartSLOduration=1.444473312 podStartE2EDuration="1.444473312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:55.442480852 +0000 UTC m=+2199.836168264" watchObservedRunningTime="2026-09-11 13:57:55.444473312 +0000 UTC m=+2199.838160723"
Sep 11 13:57:55.680865 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.680845    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:55.829710 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829678    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm9zl\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829721    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829763    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829783    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829804    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.829870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829866    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.830208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829910    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.830208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.829936    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs\") pod \"3f2e5ffd-ecc6-4986-818c-4b705708411a\" (UID: \"3f2e5ffd-ecc6-4986-818c-4b705708411a\") "
Sep 11 13:57:55.830208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.830148    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:57:55.830355 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.830236    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:57:55.830412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.830352    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:57:55.830412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.830368    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:57:55.830547 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.830451    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:57:55.832037 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.832008    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:57:55.832175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.832080    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:57:55.832175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.832103    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:57:55.832175 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.832149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl" (OuterVolumeSpecName: "kube-api-access-nm9zl") pod "3f2e5ffd-ecc6-4986-818c-4b705708411a" (UID: "3f2e5ffd-ecc6-4986-818c-4b705708411a"). InnerVolumeSpecName "kube-api-access-nm9zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:57:55.931140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931071    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931096    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931110    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm9zl\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-kube-api-access-nm9zl\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931123    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f2e5ffd-ecc6-4986-818c-4b705708411a-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931140 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931136    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931148    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931160    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931171    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f2e5ffd-ecc6-4986-818c-4b705708411a-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:55.931378 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:55.931182    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f2e5ffd-ecc6-4986-818c-4b705708411a-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:56.300245 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.300216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:56.429970 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.429937    2576 generic.go:356] "Generic (PLEG): container finished" podID="dca04d05-1c08-498d-9889-8a1b9c5007e2" containerID="1f9002238031c58bf9f9d8c91f1dcef560358a04798e4b77075895bbe39ae185" exitCode=0
Sep 11 13:57:56.430370 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.430018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8" event={"ID":"dca04d05-1c08-498d-9889-8a1b9c5007e2","Type":"ContainerDied","Data":"1f9002238031c58bf9f9d8c91f1dcef560358a04798e4b77075895bbe39ae185"}
Sep 11 13:57:56.431694 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.431669    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerID="868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb" exitCode=0
Sep 11 13:57:56.431819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.431711    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"
Sep 11 13:57:56.431819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.431752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerDied","Data":"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"}
Sep 11 13:57:56.431819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.431794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr" event={"ID":"3f2e5ffd-ecc6-4986-818c-4b705708411a","Type":"ContainerDied","Data":"82fe2585c87e70b86ae2b72878d1fc225693c76959f5482c88afbb0a0584ca15"}
Sep 11 13:57:56.431819 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.431816    2576 scope.go:117] "RemoveContainer" containerID="868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"
Sep 11 13:57:56.440319 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.440303    2576 scope.go:117] "RemoveContainer" containerID="cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"
Sep 11 13:57:56.448090 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.448073    2576 scope.go:117] "RemoveContainer" containerID="868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"
Sep 11 13:57:56.448340 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:57:56.448322    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb\": container with ID starting with 868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb not found: ID does not exist" containerID="868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"
Sep 11 13:57:56.448405 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.448352    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb"} err="failed to get container status \"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb\": rpc error: code = NotFound desc = could not find container \"868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb\": container with ID starting with 868c7826e50bcd83a0b2c8ff08c324fde223d834b039b71a2b6c9cbf6f27f0cb not found: ID does not exist"
Sep 11 13:57:56.448405 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.448376    2576 scope.go:117] "RemoveContainer" containerID="cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"
Sep 11 13:57:56.448670 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:57:56.448650    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce\": container with ID starting with cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce not found: ID does not exist" containerID="cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"
Sep 11 13:57:56.448725 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.448677    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce"} err="failed to get container status \"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce\": rpc error: code = NotFound desc = could not find container \"cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce\": container with ID starting with cf18e5ad6322ab285a15da97b4519c00a504f997a4c8a2a66090f7724c27c6ce not found: ID does not exist"
Sep 11 13:57:56.470608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.470584    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:57:56.473467 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:56.473443    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-7b7819-openshift-default-8597c4ffbd-zsqbr"]
Sep 11 13:57:57.184464 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:57.184430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:57.564125 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:57.564102    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:57.648772 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:57.648742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca\") pod \"dca04d05-1c08-498d-9889-8a1b9c5007e2\" (UID: \"dca04d05-1c08-498d-9889-8a1b9c5007e2\") "
Sep 11 13:57:57.649108 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:57.649085    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dca04d05-1c08-498d-9889-8a1b9c5007e2" (UID: "dca04d05-1c08-498d-9889-8a1b9c5007e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:57:57.749862 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:57.749806    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dca04d05-1c08-498d-9889-8a1b9c5007e2-maas-api-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:57:58.099524 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.099470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:57:58.248269 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.248238    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" path="/var/lib/kubelet/pods/3f2e5ffd-ecc6-4986-818c-4b705708411a/volumes"
Sep 11 13:57:58.444662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.444577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8" event={"ID":"dca04d05-1c08-498d-9889-8a1b9c5007e2","Type":"ContainerDied","Data":"0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0"}
Sep 11 13:57:58.444662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.444625    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"
Sep 11 13:57:58.444662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.444635    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d023677048297b6c3d47e16c88bcd0c84e4bf6e19bfebd2040312ba6023f9b0"
Sep 11 13:57:58.890582 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:57:58.890550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:09.928608 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:09.928577    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:09.994698 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:09.994662    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:58:09.994940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:09.994902    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" podUID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" containerName="payload-pre-processing" containerID="cri-o://1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0" gracePeriod=30
Sep 11 13:58:10.009888 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.009864    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:10.069930 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.069899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:10.173486 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.173428    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"]
Sep 11 13:58:10.173777 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.173688    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:10.175548 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.175522    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-264b08-zjkm8"]
Sep 11 13:58:10.245938 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.245916    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:58:10.247533 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.247510    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dca04d05-1c08-498d-9889-8a1b9c5007e2" path="/var/lib/kubelet/pods/dca04d05-1c08-498d-9889-8a1b9c5007e2/volumes"
Sep 11 13:58:10.257601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.257578    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume\") pod \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") "
Sep 11 13:58:10.257706 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.257689    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgq2s\" (UniqueName: \"kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s\") pod \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\" (UID: \"719bb040-6e73-4ffe-b4bf-5e9f93f26796\") "
Sep 11 13:58:10.257976 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.257954    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "719bb040-6e73-4ffe-b4bf-5e9f93f26796" (UID: "719bb040-6e73-4ffe-b4bf-5e9f93f26796"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:58:10.259652 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.259626    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s" (OuterVolumeSpecName: "kube-api-access-hgq2s") pod "719bb040-6e73-4ffe-b4bf-5e9f93f26796" (UID: "719bb040-6e73-4ffe-b4bf-5e9f93f26796"). InnerVolumeSpecName "kube-api-access-hgq2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:58:10.271566 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.271545    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:10.359134 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.359092    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgq2s\" (UniqueName: \"kubernetes.io/projected/719bb040-6e73-4ffe-b4bf-5e9f93f26796-kube-api-access-hgq2s\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:58:10.359134 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.359125    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/719bb040-6e73-4ffe-b4bf-5e9f93f26796-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:58:10.498149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.498067    2576 generic.go:356] "Generic (PLEG): container finished" podID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" containerID="1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0" exitCode=0
Sep 11 13:58:10.498149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.498114    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"
Sep 11 13:58:10.498149 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.498135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" event={"ID":"719bb040-6e73-4ffe-b4bf-5e9f93f26796","Type":"ContainerDied","Data":"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"}
Sep 11 13:58:10.498371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.498178    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt" event={"ID":"719bb040-6e73-4ffe-b4bf-5e9f93f26796","Type":"ContainerDied","Data":"c45d14cabe77d6c093db5c426e5beee29fbf87f51ba732d385d48e11c4f7c10b"}
Sep 11 13:58:10.498371 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.498199    2576 scope.go:117] "RemoveContainer" containerID="1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"
Sep 11 13:58:10.506691 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.506675    2576 scope.go:117] "RemoveContainer" containerID="1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"
Sep 11 13:58:10.506935 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:58:10.506913    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0\": container with ID starting with 1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0 not found: ID does not exist" containerID="1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"
Sep 11 13:58:10.506984 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.506944    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0"} err="failed to get container status \"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0\": rpc error: code = NotFound desc = could not find container \"1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0\": container with ID starting with 1b39f61b8fee0c2bc8731b658c839b6183daf12445460ad869ac04667586e4f0 not found: ID does not exist"
Sep 11 13:58:10.520062 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.520040    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:58:10.524685 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.524663    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-264b08-5dd9bbfb6-9xjqt"]
Sep 11 13:58:10.601012 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.600979    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:10.701553 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:10.701525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:11.465518 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:11.465466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:12.249014 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:12.248979    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" path="/var/lib/kubelet/pods/719bb040-6e73-4ffe-b4bf-5e9f93f26796/volumes"
Sep 11 13:58:12.344386 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:12.344354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:13.136662 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:13.136632    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:15.675723 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:15.675693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:16.490007 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:16.489977    2576 scope.go:117] "RemoveContainer" containerID="b2ba672f2dfffebb87904a8406f37c4b123b77297c0d674b327e6240624f9c6f"
Sep 11 13:58:48.158544 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:48.158459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:49.420361 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:49.420330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:55.173128 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:55.173048    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:55.232967 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:55.232939    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:55.483860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:55.483780    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:56.185599 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:56.185565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:57.077969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:57.077941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:57.898987 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:57.898958    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:58.855368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:58:58.855323    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:00.019684 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:00.019645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:03.483022 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:03.482990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:06.760901 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:06.760858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:06.983368 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:06.983333    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:07.095257 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:07.095227    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:07.292974 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:07.292942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:07.370293 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:07.370229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:07.897774 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:07.897743    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:08.009620 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:08.009592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:09.250393 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:09.250360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:10.759697 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:10.759662    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:14.494483 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:14.494448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:14.613276 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:14.613247    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:14.821447 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:14.821416    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:59:14.821664 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:14.821643    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy" containerID="cri-o://496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733" gracePeriod=30
Sep 11 13:59:14.835082 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:14.835057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:16.105138 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:16.105102    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:17.163265 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:17.163229    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:17.976310 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:17.976276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:20.065208 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.065186    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:59:20.178100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178017    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178100 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178069    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178124    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178214    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178256    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178308 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178294    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndqk9\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178330    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178375    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs\") pod \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\" (UID: \"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed\") "
Sep 11 13:59:20.178609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178397    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:20.178609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:20.178609 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178483    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:20.178835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178604    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:20.178835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178705    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:20.178835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178719    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.178835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178739    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.178835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.178754    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.180320 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.180299    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:20.180417 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.180333    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:20.180826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.180808    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:59:20.180963 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.180825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9" (OuterVolumeSpecName: "kube-api-access-ndqk9") pod "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" (UID: "9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed"). InnerVolumeSpecName "kube-api-access-ndqk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:20.279792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279766    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.279792 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279788    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.279926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279797    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndqk9\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-kube-api-access-ndqk9\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.279926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279806    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.279926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279814    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.279926 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.279823    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:20.823702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.823666    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerID="496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733" exitCode=0
Sep 11 13:59:20.823702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.823706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerDied","Data":"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"}
Sep 11 13:59:20.823915 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.823718    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"
Sep 11 13:59:20.823915 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.823778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt" event={"ID":"9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed","Type":"ContainerDied","Data":"fd414f23f9eadb13b6d3e2a022cdfb8bf280db2860eb8f902fa1bc8266c93533"}
Sep 11 13:59:20.823915 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.823801    2576 scope.go:117] "RemoveContainer" containerID="496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"
Sep 11 13:59:20.832010 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.831988    2576 scope.go:117] "RemoveContainer" containerID="5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"
Sep 11 13:59:20.839403 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.839385    2576 scope.go:117] "RemoveContainer" containerID="496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"
Sep 11 13:59:20.839670 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:20.839651    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733\": container with ID starting with 496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733 not found: ID does not exist" containerID="496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"
Sep 11 13:59:20.839746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.839682    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733"} err="failed to get container status \"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733\": rpc error: code = NotFound desc = could not find container \"496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733\": container with ID starting with 496e7ee248cf498192a871977da8f86fe3257c169bec14aa58297f649c228733 not found: ID does not exist"
Sep 11 13:59:20.839746 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.839698    2576 scope.go:117] "RemoveContainer" containerID="5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"
Sep 11 13:59:20.839841 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.839778    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:59:20.839944 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:20.839928    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45\": container with ID starting with 5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45 not found: ID does not exist" containerID="5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"
Sep 11 13:59:20.839986 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.839947    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45"} err="failed to get container status \"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45\": rpc error: code = NotFound desc = could not find container \"5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45\": container with ID starting with 5b5267e068ddf93c78733a8a1efe9075f2e2535c920aa03cb42c83d45c969e45 not found: ID does not exist"
Sep 11 13:59:20.845835 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:20.845814    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-a403bc-openshift-default-6b77b9ff66-2c4lt"]
Sep 11 13:59:22.248457 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:22.248428    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" path="/var/lib/kubelet/pods/9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed/volumes"
Sep 11 13:59:25.841797 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:25.841762    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:25.901885 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:25.901849    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:59:25.902139 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:25.902114    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" podUID="052bebd0-f531-4be3-9548-f2f7ffe007f8" containerName="payload-pre-processing" containerID="cri-o://0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192" gracePeriod=30
Sep 11 13:59:25.933260 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:25.933235    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:26.034757 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.034730    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:26.096990 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.096924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:26.144860 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.144835    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:59:26.227484 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.227452    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7pq2\" (UniqueName: \"kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2\") pod \"052bebd0-f531-4be3-9548-f2f7ffe007f8\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") "
Sep 11 13:59:26.227683 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.227572    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume\") pod \"052bebd0-f531-4be3-9548-f2f7ffe007f8\" (UID: \"052bebd0-f531-4be3-9548-f2f7ffe007f8\") "
Sep 11 13:59:26.227939 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.227915    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "052bebd0-f531-4be3-9548-f2f7ffe007f8" (UID: "052bebd0-f531-4be3-9548-f2f7ffe007f8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:26.229680 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.229648    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2" (OuterVolumeSpecName: "kube-api-access-q7pq2") pod "052bebd0-f531-4be3-9548-f2f7ffe007f8" (UID: "052bebd0-f531-4be3-9548-f2f7ffe007f8"). InnerVolumeSpecName "kube-api-access-q7pq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:26.328640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.328600    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/052bebd0-f531-4be3-9548-f2f7ffe007f8-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:26.328640 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.328639    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7pq2\" (UniqueName: \"kubernetes.io/projected/052bebd0-f531-4be3-9548-f2f7ffe007f8-kube-api-access-q7pq2\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:26.517936 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.517908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:26.617969 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.617933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:26.854541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.854430    2576 generic.go:356] "Generic (PLEG): container finished" podID="052bebd0-f531-4be3-9548-f2f7ffe007f8" containerID="0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192" exitCode=0
Sep 11 13:59:26.854541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.854469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" event={"ID":"052bebd0-f531-4be3-9548-f2f7ffe007f8","Type":"ContainerDied","Data":"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"}
Sep 11 13:59:26.854541 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.854488    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"
Sep 11 13:59:26.855086 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.854565    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q" event={"ID":"052bebd0-f531-4be3-9548-f2f7ffe007f8","Type":"ContainerDied","Data":"9279612c935b30872e2370cb7142eca84683d7f65343ed8fc3edcdd742bdf214"}
Sep 11 13:59:26.855086 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.854590    2576 scope.go:117] "RemoveContainer" containerID="0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"
Sep 11 13:59:26.863654 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.863617    2576 scope.go:117] "RemoveContainer" containerID="0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"
Sep 11 13:59:26.864936 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:26.864914    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192\": container with ID starting with 0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192 not found: ID does not exist" containerID="0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"
Sep 11 13:59:26.865126 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.865100    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192"} err="failed to get container status \"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192\": rpc error: code = NotFound desc = could not find container \"0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192\": container with ID starting with 0273c845543e5b25f2e2683afea59d0ede21e6668c65bc088119b0c205515192 not found: ID does not exist"
Sep 11 13:59:26.877110 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.876882    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:59:26.879129 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:26.879107    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-0317c3-c699c6dcd-v4j6q"]
Sep 11 13:59:27.566285 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:27.566250    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:28.247406 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:28.247370    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="052bebd0-f531-4be3-9548-f2f7ffe007f8" path="/var/lib/kubelet/pods/052bebd0-f531-4be3-9548-f2f7ffe007f8/volumes"
Sep 11 13:59:33.396922 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:33.396894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:33.481610 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:33.481581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:33.757702 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:33.757622    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:59:33.757880 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:33.757854    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" containerID="cri-o://6d9ac25b9761f9651bba1a7e930abf59d5054fc442adbd7ef75ae1caad4805a7" gracePeriod=30
Sep 11 13:59:33.787973 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:33.787949    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:34.408200 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:34.408171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:35.284882 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:35.284849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:36.186031 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:36.186004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:37.517162 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:37.517127    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:59:37.517556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:37.517195    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:59:38.910412 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:38.910378    2576 generic.go:356] "Generic (PLEG): container finished" podID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerID="6d9ac25b9761f9651bba1a7e930abf59d5054fc442adbd7ef75ae1caad4805a7" exitCode=0
Sep 11 13:59:38.910799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:38.910455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerDied","Data":"6d9ac25b9761f9651bba1a7e930abf59d5054fc442adbd7ef75ae1caad4805a7"}
Sep 11 13:59:38.910799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:38.910549    2576 scope.go:117] "RemoveContainer" containerID="7362ac0ef5efb001ea2c007f636436db28d3b027be0e0a8863f1622a20bc571c"
Sep 11 13:59:39.003227 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.003203    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:59:39.037661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037577    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037661 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037635    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msw75\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037727    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037833    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:39.037929 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.037881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data\") pod \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\" (UID: \"62b74d74-1b5e-45d5-9a3c-8b78900521b2\") "
Sep 11 13:59:39.038375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:39.038375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038219    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.038375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038237    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.038375 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038285    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:39.038628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038451    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:39.038628 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.038469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:39.039999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.039954    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:39.039999 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.039989    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:59:39.040318 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.040297    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:39.040382 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.040331    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75" (OuterVolumeSpecName: "kube-api-access-msw75") pod "62b74d74-1b5e-45d5-9a3c-8b78900521b2" (UID: "62b74d74-1b5e-45d5-9a3c-8b78900521b2"). InnerVolumeSpecName "kube-api-access-msw75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:39.138778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138733    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.138778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138772    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.138778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138781    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msw75\" (UniqueName: \"kubernetes.io/projected/62b74d74-1b5e-45d5-9a3c-8b78900521b2-kube-api-access-msw75\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.138778 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138790    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.139043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138799    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.139043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138807    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.139043 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.138815    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62b74d74-1b5e-45d5-9a3c-8b78900521b2-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:39.920799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.920762    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"
Sep 11 13:59:39.920799 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.920781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8" event={"ID":"62b74d74-1b5e-45d5-9a3c-8b78900521b2","Type":"ContainerDied","Data":"573de0aa9c05d179e9c024f826263ce1646f806c745abe2a57ed1d14b9149990"}
Sep 11 13:59:39.921278 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.920918    2576 scope.go:117] "RemoveContainer" containerID="6d9ac25b9761f9651bba1a7e930abf59d5054fc442adbd7ef75ae1caad4805a7"
Sep 11 13:59:39.942280 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.942253    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:59:39.947964 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:39.947933    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-0317c3-openshift-default-887f46f6b-mh6d8"]
Sep 11 13:59:40.247985 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:40.247899    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" path="/var/lib/kubelet/pods/62b74d74-1b5e-45d5-9a3c-8b78900521b2/volumes"
Sep 11 13:59:44.901636 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:44.901602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:44.907011 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:44.906987    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"]
Sep 11 13:59:44.909123 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:44.909098    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-9edb53-29818905-gm8wg"]
Sep 11 13:59:44.938556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:44.938526    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:59:44.938880 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:44.938854    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" podUID="52034605-5fcd-4899-aa52-4654431bb09e" containerName="payload-pre-processing" containerID="cri-o://8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4" gracePeriod=30
Sep 11 13:59:45.000284 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.000259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:45.100472 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.100443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:45.167556 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.167466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:45.192252 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.192229    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:59:45.282870 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.282424    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume\") pod \"52034605-5fcd-4899-aa52-4654431bb09e\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") "
Sep 11 13:59:45.283051 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.282929    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52034605-5fcd-4899-aa52-4654431bb09e" (UID: "52034605-5fcd-4899-aa52-4654431bb09e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:45.283135 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.283059    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg8zw\" (UniqueName: \"kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw\") pod \"52034605-5fcd-4899-aa52-4654431bb09e\" (UID: \"52034605-5fcd-4899-aa52-4654431bb09e\") "
Sep 11 13:59:45.283528 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.283483    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52034605-5fcd-4899-aa52-4654431bb09e-plugins-config-volume\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:45.285921 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.285890    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw" (OuterVolumeSpecName: "kube-api-access-jg8zw") pod "52034605-5fcd-4899-aa52-4654431bb09e" (UID: "52034605-5fcd-4899-aa52-4654431bb09e"). InnerVolumeSpecName "kube-api-access-jg8zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:45.384525 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.384449    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg8zw\" (UniqueName: \"kubernetes.io/projected/52034605-5fcd-4899-aa52-4654431bb09e-kube-api-access-jg8zw\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:45.525419 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.525387    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:45.592296 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.592261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:45.951275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.951165    2576 generic.go:356] "Generic (PLEG): container finished" podID="52034605-5fcd-4899-aa52-4654431bb09e" containerID="8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4" exitCode=0
Sep 11 13:59:45.951275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.951212    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"
Sep 11 13:59:45.951275 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.951254    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" event={"ID":"52034605-5fcd-4899-aa52-4654431bb09e","Type":"ContainerDied","Data":"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"}
Sep 11 13:59:45.951855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.951310    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j" event={"ID":"52034605-5fcd-4899-aa52-4654431bb09e","Type":"ContainerDied","Data":"ead2d807df3ccc9b38566a05a91fc16174dcf8ae13867eb20d45c7fb7c6faeb4"}
Sep 11 13:59:45.951855 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.951330    2576 scope.go:117] "RemoveContainer" containerID="8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"
Sep 11 13:59:45.967388 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.967370    2576 scope.go:117] "RemoveContainer" containerID="8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"
Sep 11 13:59:45.967689 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:45.967664    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4\": container with ID starting with 8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4 not found: ID does not exist" containerID="8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"
Sep 11 13:59:45.967822 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.967694    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4"} err="failed to get container status \"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4\": rpc error: code = NotFound desc = could not find container \"8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4\": container with ID starting with 8d03a2421b7512393f68510babcb71ae41d3ad9c7b47116fdb5682d686d254b4 not found: ID does not exist"
Sep 11 13:59:45.977770 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.977748    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:59:45.986950 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:45.986924    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9edb53-7b4c6f8bd6-qld8j"]
Sep 11 13:59:46.247652 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:46.247575    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eeb6c2a-f8a5-49a4-9c53-141968bccc32" path="/var/lib/kubelet/pods/1eeb6c2a-f8a5-49a4-9c53-141968bccc32/volumes"
Sep 11 13:59:46.247904 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:46.247891    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52034605-5fcd-4899-aa52-4654431bb09e" path="/var/lib/kubelet/pods/52034605-5fcd-4899-aa52-4654431bb09e/volumes"
Sep 11 13:59:52.774861 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:52.774821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:52.843766 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:52.843731    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:53.069655 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:53.069576    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 13:59:53.069822 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:53.069786    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy" containerID="cri-o://bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a" gracePeriod=30
Sep 11 13:59:53.081634 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:53.081610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 13:59:58.310867 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.310839    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:59:58.393042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.392951    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.392991    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393042 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393023    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393091    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtl59\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393207    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393271    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert\") pod \"9e24c047-5ccf-4f37-8cb3-9963f173544d\" (UID: \"9e24c047-5ccf-4f37-8cb3-9963f173544d\") "
Sep 11 13:59:58.393306 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393274    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393460    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393672    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393683    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393733    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-data\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393791    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:58.393940 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.393806    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:59:58.395357 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.395329    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59" (OuterVolumeSpecName: "kube-api-access-xtl59") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "kube-api-access-xtl59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:58.395601 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.395548    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:59:58.395826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.395801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:59:58.395826 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.395812    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9e24c047-5ccf-4f37-8cb3-9963f173544d" (UID: "9e24c047-5ccf-4f37-8cb3-9963f173544d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:59:58.494954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494910    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-envoy\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.494954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494949    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-credential-socket\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.494954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494960    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e24c047-5ccf-4f37-8cb3-9963f173544d-istiod-ca-cert\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.494954 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494970    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-token\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.495216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494978    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e24c047-5ccf-4f37-8cb3-9963f173544d-workload-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.495216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494986    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtl59\" (UniqueName: \"kubernetes.io/projected/9e24c047-5ccf-4f37-8cb3-9963f173544d-kube-api-access-xtl59\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:58.495216 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:58.494994    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e24c047-5ccf-4f37-8cb3-9963f173544d-istio-podinfo\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 13:59:59.010245 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.010212    2576 generic.go:356] "Generic (PLEG): container finished" podID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerID="bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a" exitCode=0
Sep 11 13:59:59.010394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.010270    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"
Sep 11 13:59:59.010394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.010293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerDied","Data":"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"}
Sep 11 13:59:59.010394 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.010374    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp" event={"ID":"9e24c047-5ccf-4f37-8cb3-9963f173544d","Type":"ContainerDied","Data":"70025309ecb83856af5fb5feb35f341e5654af7c050202bf40f56a2fe98ae0c8"}
Sep 11 13:59:59.010572 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.010397    2576 scope.go:117] "RemoveContainer" containerID="bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"
Sep 11 13:59:59.018782 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.018759    2576 scope.go:117] "RemoveContainer" containerID="0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"
Sep 11 13:59:59.027908 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.027890    2576 scope.go:117] "RemoveContainer" containerID="bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"
Sep 11 13:59:59.028165 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:59.028147    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a\": container with ID starting with bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a not found: ID does not exist" containerID="bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"
Sep 11 13:59:59.028232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.028170    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a"} err="failed to get container status \"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a\": rpc error: code = NotFound desc = could not find container \"bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a\": container with ID starting with bc62b00600f30ef752d127593123e4c7bbc3896265e833a2af74bf7de1c71f6a not found: ID does not exist"
Sep 11 13:59:59.028232 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.028185    2576 scope.go:117] "RemoveContainer" containerID="0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"
Sep 11 13:59:59.028422 ip-10-0-133-221 kubenswrapper[2576]: E0911 13:59:59.028402    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1\": container with ID starting with 0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1 not found: ID does not exist" containerID="0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"
Sep 11 13:59:59.028460 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.028429    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1"} err="failed to get container status \"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1\": rpc error: code = NotFound desc = could not find container \"0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1\": container with ID starting with 0dd87e90d404b3578bbfa4a812dae7d2af9e342fe82dfcadf0f7f3f3341155c1 not found: ID does not exist"
Sep 11 13:59:59.036843 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.036822    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 13:59:59.042009 ip-10-0-133-221 kubenswrapper[2576]: I0911 13:59:59.041991    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9edb53-openshift-default-55bb4dddc4-6wjnp"]
Sep 11 14:00:00.145833 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.145782    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"]
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146198    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146216    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146231    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146239    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146249    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146258    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146274    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="052bebd0-f531-4be3-9548-f2f7ffe007f8" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146282    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="052bebd0-f531-4be3-9548-f2f7ffe007f8" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146291    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146298    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146309    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52034605-5fcd-4899-aa52-4654431bb09e" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146316    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="52034605-5fcd-4899-aa52-4654431bb09e" containerName="payload-pre-processing"
Sep 11 14:00:00.146328 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146328    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146336    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146345    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca04d05-1c08-498d-9889-8a1b9c5007e2" containerName="revoke-keys"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146353    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca04d05-1c08-498d-9889-8a1b9c5007e2" containerName="revoke-keys"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146369    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146378    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146387    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146395    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146407    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146414    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146491    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146521    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="052bebd0-f531-4be3-9548-f2f7ffe007f8" containerName="payload-pre-processing"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146532    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146542    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146551    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="719bb040-6e73-4ffe-b4bf-5e9f93f26796" containerName="payload-pre-processing"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146559    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca04d05-1c08-498d-9889-8a1b9c5007e2" containerName="revoke-keys"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146569    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="52034605-5fcd-4899-aa52-4654431bb09e" containerName="payload-pre-processing"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146577    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.147002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.146586    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f2e5ffd-ecc6-4986-818c-4b705708411a" containerName="istio-proxy"
Sep 11 14:00:00.149674 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.149653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"
Sep 11 14:00:00.153246 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.153229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lxn7r\""
Sep 11 14:00:00.159424 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.159404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"
Sep 11 14:00:00.160780 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.160755    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"]
Sep 11 14:00:00.249089 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.249047    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" path="/var/lib/kubelet/pods/9e24c047-5ccf-4f37-8cb3-9963f173544d/volumes"
Sep 11 14:00:00.429743 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:00.429716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"]
Sep 11 14:00:00.431625 ip-10-0-133-221 kubenswrapper[2576]: W0911 14:00:00.431601    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529ee338_916e_40aa_be46_80a790bb89bb.slice/crio-251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0 WatchSource:0}: Error finding container 251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0: Status 404 returned error can't find the container with id 251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0
Sep 11 14:00:01.021405 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:01.021370    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh" event={"ID":"529ee338-916e-40aa-be46-80a790bb89bb","Type":"ContainerStarted","Data":"4b1e9f2c2672357e9f519ff56f71ee5134118c9ab32fa5e5080c0f384029bf5c"}
Sep 11 14:00:01.021599 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:01.021426    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh" event={"ID":"529ee338-916e-40aa-be46-80a790bb89bb","Type":"ContainerStarted","Data":"251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0"}
Sep 11 14:00:01.037907 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:01.037854    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh" podStartSLOduration=1.03783613 podStartE2EDuration="1.03783613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:00:01.037392434 +0000 UTC m=+2325.431079845" watchObservedRunningTime="2026-09-11 14:00:01.03783613 +0000 UTC m=+2325.431523543"
Sep 11 14:00:01.593451 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:01.593363    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:00:02.027332 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:02.027298    2576 generic.go:356] "Generic (PLEG): container finished" podID="529ee338-916e-40aa-be46-80a790bb89bb" containerID="4b1e9f2c2672357e9f519ff56f71ee5134118c9ab32fa5e5080c0f384029bf5c" exitCode=0
Sep 11 14:00:02.027544 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:02.027389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh" event={"ID":"529ee338-916e-40aa-be46-80a790bb89bb","Type":"ContainerDied","Data":"4b1e9f2c2672357e9f519ff56f71ee5134118c9ab32fa5e5080c0f384029bf5c"}
Sep 11 14:00:03.154113 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:03.154088    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"
Sep 11 14:00:04.037072 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:04.037035    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh"
Sep 11 14:00:04.037072 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:04.037064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29818920-gkpwh" event={"ID":"529ee338-916e-40aa-be46-80a790bb89bb","Type":"ContainerDied","Data":"251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0"}
Sep 11 14:00:04.037292 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:04.037115    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="251ed16c230349f994c47a523e474d2c1dbdda2f0661b7b2fb7293199f386fe0"
Sep 11 14:00:04.945991 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:04.945963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:00:16.523900 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:00:16.523832    2576 scope.go:117] "RemoveContainer" containerID="3a5a6ce67b6e583ea9c0538fcf20a7434de5ccff8354909af6a17b016028d3c4"
Sep 11 14:01:17.447962 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:01:17.447927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:01:19.138515 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:01:19.138469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:01:44.543055 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:01:44.542976    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:02:06.143606 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:02:06.143575    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:02:32.847136 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:02:32.847101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:02:32.942323 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:02:32.942291    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:02:50.053958 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:02:50.053899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:03:16.569834 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:03:16.569749    2576 scope.go:117] "RemoveContainer" containerID="cfea7e18ecd8906cbb92d27f7c341ad569156ee6174a27c7d1706d6eba6c9669"
Sep 11 14:04:12.089783 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:12.089752    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:04:16.586079 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:16.586046    2576 scope.go:117] "RemoveContainer" containerID="1f9002238031c58bf9f9d8c91f1dcef560358a04798e4b77075895bbe39ae185"
Sep 11 14:04:16.593733 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:16.593707    2576 scope.go:117] "RemoveContainer" containerID="f38106ec293b46e5eb2c646fd4d20f3148aede6a5dd1f5af5c5ae5181eeef0ff"
Sep 11 14:04:20.561542 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:20.561506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:04:42.656632 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:42.656551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:04:44.053767 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:04:44.053729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:05:02.580586 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580545    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580847    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580860    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580874    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="529ee338-916e-40aa-be46-80a790bb89bb" containerName="cleanup"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580880    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="529ee338-916e-40aa-be46-80a790bb89bb" containerName="cleanup"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580934    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b2e53bc-f8cd-444c-bf15-f9bc4f5689ed" containerName="istio-proxy"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580945    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e24c047-5ccf-4f37-8cb3-9963f173544d" containerName="istio-proxy"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580954    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="62b74d74-1b5e-45d5-9a3c-8b78900521b2" containerName="istio-proxy"
Sep 11 14:05:02.580996 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.580961    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="529ee338-916e-40aa-be46-80a790bb89bb" containerName="cleanup"
Sep 11 14:05:02.583780 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.583764    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.588739 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.588536    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6zc5p\""
Sep 11 14:05:02.588739 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.588568    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 14:05:02.588739 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.588584    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 14:05:02.588739 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.588597    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 14:05:02.589403 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.589383    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:05:02.606239 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.606214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:02.663539 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.663483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cv5\" (UniqueName: \"kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.663707 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.663574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.663707 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.663615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.663707 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.663679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.764968 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.764934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cv5\" (UniqueName: \"kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.765173 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.764988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.765173 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.765017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.765173 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.765044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.765639 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.765615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.767567 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.767548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.767632 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.767574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.774255 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.774229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cv5\" (UniqueName: \"kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5\") pod \"maas-api-5d987ff8b5-m5cms\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") " pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:02.895547 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:02.895425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:03.172632 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:03.172484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:03.175272 ip-10-0-133-221 kubenswrapper[2576]: W0911 14:05:03.175241    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d008944_53dd_4022_8fec_238fdad55bdc.slice/crio-a4d4aec2e3b857b5bdc7f06dde54d817c1775b75f057f4ea149c5390764c5619 WatchSource:0}: Error finding container a4d4aec2e3b857b5bdc7f06dde54d817c1775b75f057f4ea149c5390764c5619: Status 404 returned error can't find the container with id a4d4aec2e3b857b5bdc7f06dde54d817c1775b75f057f4ea149c5390764c5619
Sep 11 14:05:03.176465 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:03.176447    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:05:03.278041 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:03.278007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" event={"ID":"3d008944-53dd-4022-8fec-238fdad55bdc","Type":"ContainerStarted","Data":"a4d4aec2e3b857b5bdc7f06dde54d817c1775b75f057f4ea149c5390764c5619"}
Sep 11 14:05:06.292968 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:06.292935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" event={"ID":"3d008944-53dd-4022-8fec-238fdad55bdc","Type":"ContainerStarted","Data":"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"}
Sep 11 14:05:06.293347 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:06.293062    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:06.312687 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:06.312628    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" podStartSLOduration=1.897901606 podStartE2EDuration="4.312612935s" totalImagesPullingTime="2.414711329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:06.311532115 +0000 UTC m=+2630.705219524" watchObservedRunningTime="2026-09-11 14:05:06.312612935 +0000 UTC m=+2630.706300345"
Sep 11 14:05:10.327594 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.327561    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:10.328033 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.327720    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" podUID="3d008944-53dd-4022-8fec-238fdad55bdc" containerName="maas-api" containerID="cri-o://a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172" gracePeriod=30
Sep 11 14:05:10.332663 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.332638    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:10.567786 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.567763    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:10.634158 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.634088    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs\") pod \"3d008944-53dd-4022-8fec-238fdad55bdc\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") "
Sep 11 14:05:10.634158 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.634133    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls\") pod \"3d008944-53dd-4022-8fec-238fdad55bdc\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") "
Sep 11 14:05:10.634158 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.634156    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5cv5\" (UniqueName: \"kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5\") pod \"3d008944-53dd-4022-8fec-238fdad55bdc\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") "
Sep 11 14:05:10.634406 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.634273    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca\") pod \"3d008944-53dd-4022-8fec-238fdad55bdc\" (UID: \"3d008944-53dd-4022-8fec-238fdad55bdc\") "
Sep 11 14:05:10.634678 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.634653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d008944-53dd-4022-8fec-238fdad55bdc" (UID: "3d008944-53dd-4022-8fec-238fdad55bdc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:05:10.636350 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.636318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5" (OuterVolumeSpecName: "kube-api-access-z5cv5") pod "3d008944-53dd-4022-8fec-238fdad55bdc" (UID: "3d008944-53dd-4022-8fec-238fdad55bdc"). InnerVolumeSpecName "kube-api-access-z5cv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:05:10.636458 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.636340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d008944-53dd-4022-8fec-238fdad55bdc" (UID: "3d008944-53dd-4022-8fec-238fdad55bdc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:10.636458 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.636364    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3d008944-53dd-4022-8fec-238fdad55bdc" (UID: "3d008944-53dd-4022-8fec-238fdad55bdc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:05:10.735637 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.735612    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-metrics-certs\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 14:05:10.735637 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.735640    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d008944-53dd-4022-8fec-238fdad55bdc-maas-api-tls\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 14:05:10.735788 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.735656    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5cv5\" (UniqueName: \"kubernetes.io/projected/3d008944-53dd-4022-8fec-238fdad55bdc-kube-api-access-z5cv5\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 14:05:10.735788 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:10.735669    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d008944-53dd-4022-8fec-238fdad55bdc-openshift-service-ca\") on node \"ip-10-0-133-221.ec2.internal\" DevicePath \"\""
Sep 11 14:05:11.314587 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.314552    2576 generic.go:356] "Generic (PLEG): container finished" podID="3d008944-53dd-4022-8fec-238fdad55bdc" containerID="a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172" exitCode=0
Sep 11 14:05:11.314755 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.314594    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"
Sep 11 14:05:11.314755 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.314636    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" event={"ID":"3d008944-53dd-4022-8fec-238fdad55bdc","Type":"ContainerDied","Data":"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"}
Sep 11 14:05:11.314755 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.314684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms" event={"ID":"3d008944-53dd-4022-8fec-238fdad55bdc","Type":"ContainerDied","Data":"a4d4aec2e3b857b5bdc7f06dde54d817c1775b75f057f4ea149c5390764c5619"}
Sep 11 14:05:11.314755 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.314706    2576 scope.go:117] "RemoveContainer" containerID="a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"
Sep 11 14:05:11.323420 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.323403    2576 scope.go:117] "RemoveContainer" containerID="a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"
Sep 11 14:05:11.323695 ip-10-0-133-221 kubenswrapper[2576]: E0911 14:05:11.323676    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172\": container with ID starting with a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172 not found: ID does not exist" containerID="a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"
Sep 11 14:05:11.323773 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.323703    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172"} err="failed to get container status \"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172\": rpc error: code = NotFound desc = could not find container \"a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172\": container with ID starting with a9f5ff2c89139c618b32da79f96e20723f5c85af55f2f09e36b5b65746072172 not found: ID does not exist"
Sep 11 14:05:11.339425 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.339401    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:11.342561 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:11.342541    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5d987ff8b5-m5cms"]
Sep 11 14:05:12.248193 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:12.248157    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d008944-53dd-4022-8fec-238fdad55bdc" path="/var/lib/kubelet/pods/3d008944-53dd-4022-8fec-238fdad55bdc/volumes"
Sep 11 14:05:35.093064 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:35.093030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:05:35.193567 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:35.193542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:05:40.234721 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:40.234690    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v4p74_6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa/server/0.log"
Sep 11 14:05:40.383583 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:40.383555    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kv2sr_4165f24d-343b-4bb9-8dd9-354b7e639e71/manager/0.log"
Sep 11 14:05:40.646605 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:40.646577    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-hc9dj_b2e85ee5-6a8e-47a6-b66c-fa554515c01b/copy-manifests/0.log"
Sep 11 14:05:40.659546 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:40.659526    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-hc9dj_b2e85ee5-6a8e-47a6-b66c-fa554515c01b/manager/0.log"
Sep 11 14:05:41.639078 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:41.639052    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29818920-gkpwh_529ee338-916e-40aa-be46-80a790bb89bb/cleanup/0.log"
Sep 11 14:05:43.934944 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:43.934915    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6_baacab4d-9cb0-46af-bc3f-a65f7ce4d74b/istio-proxy/0.log"
Sep 11 14:05:44.061674 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:44.061646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-shkfw_e6491d9c-b73d-481a-8c82-f16de3694df4/discovery/0.log"
Sep 11 14:05:45.379673 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:45.379644    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8n4rq_d41a4e3b-8a4c-4722-ae70-0da2325a2b29/main/0.log"
Sep 11 14:05:45.514348 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:45.514318    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-f7w6x_e8c42d78-84ad-469d-9187-5b7f70f5e6fe/main/0.log"
Sep 11 14:05:45.914434 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:45.914403    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fmxvjm_65e6a84f-fe78-4af0-8526-7e16d41ca986/main/0.log"
Sep 11 14:05:53.026615 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:53.026583    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:05:53.123247 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:53.123215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log" limit=1024
Sep 11 14:05:56.261764 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:56.261729    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-crjpf_45c1f894-db5d-445d-a062-41f989520fa2/global-pull-secret-syncer/0.log"
Sep 11 14:05:56.436185 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:56.436153    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nvmtf_3fab8561-db61-40c9-9972-73e8e638cc7f/konnectivity-agent/0.log"
Sep 11 14:05:56.592214 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:05:56.592190    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-221.ec2.internal_e834e13edb4b1c98f5d9c9a73bedc2fb/haproxy/0.log"
Sep 11 14:06:03.443570 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:03.443477    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlcgl_4fb00de1-7ea5-4a4e-b917-a2711107b7f2/node-exporter/0.log"
Sep 11 14:06:03.467803 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:03.467774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlcgl_4fb00de1-7ea5-4a4e-b917-a2711107b7f2/kube-rbac-proxy/0.log"
Sep 11 14:06:03.492390 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:03.492360    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hlcgl_4fb00de1-7ea5-4a4e-b917-a2711107b7f2/init-textfile/0.log"
Sep 11 14:06:06.323246 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.323217    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"]
Sep 11 14:06:06.323629 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.323519    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d008944-53dd-4022-8fec-238fdad55bdc" containerName="maas-api"
Sep 11 14:06:06.323629 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.323535    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d008944-53dd-4022-8fec-238fdad55bdc" containerName="maas-api"
Sep 11 14:06:06.323629 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.323596    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d008944-53dd-4022-8fec-238fdad55bdc" containerName="maas-api"
Sep 11 14:06:06.326301 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.326285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.331177 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.331153    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwnld\"/\"openshift-service-ca.crt\""
Sep 11 14:06:06.331318 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.331232    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwnld\"/\"kube-root-ca.crt\""
Sep 11 14:06:06.331318 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.331260    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jwnld\"/\"default-dockercfg-zb6z7\""
Sep 11 14:06:06.347339 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.347315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"]
Sep 11 14:06:06.473556 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.473528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-sys\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.473711 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.473571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-lib-modules\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.473711 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.473601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-podres\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.473711 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.473681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-proc\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.473836 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.473713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dgm9\" (UniqueName: \"kubernetes.io/projected/5df01421-67d6-4d46-ba73-dd1ab679a59b-kube-api-access-7dgm9\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574529 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-proc\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574529 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574480    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dgm9\" (UniqueName: \"kubernetes.io/projected/5df01421-67d6-4d46-ba73-dd1ab679a59b-kube-api-access-7dgm9\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574529 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-sys\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574561    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-lib-modules\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-proc\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-podres\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574618    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-sys\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-podres\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.574722 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.574691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5df01421-67d6-4d46-ba73-dd1ab679a59b-lib-modules\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.588967 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.588942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dgm9\" (UniqueName: \"kubernetes.io/projected/5df01421-67d6-4d46-ba73-dd1ab679a59b-kube-api-access-7dgm9\") pod \"perf-node-gather-daemonset-77676\" (UID: \"5df01421-67d6-4d46-ba73-dd1ab679a59b\") " pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:06.635570 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:06.635548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:07.502427 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:07.502396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"]
Sep 11 14:06:07.506008 ip-10-0-133-221 kubenswrapper[2576]: W0911 14:06:07.505983    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5df01421_67d6_4d46_ba73_dd1ab679a59b.slice/crio-71d22c0ca6e1ce51a41f5e51d672afa43aec8ee7dbdb453fb75538c69dd388e2 WatchSource:0}: Error finding container 71d22c0ca6e1ce51a41f5e51d672afa43aec8ee7dbdb453fb75538c69dd388e2: Status 404 returned error can't find the container with id 71d22c0ca6e1ce51a41f5e51d672afa43aec8ee7dbdb453fb75538c69dd388e2
Sep 11 14:06:07.533474 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:07.533446    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676" event={"ID":"5df01421-67d6-4d46-ba73-dd1ab679a59b","Type":"ContainerStarted","Data":"71d22c0ca6e1ce51a41f5e51d672afa43aec8ee7dbdb453fb75538c69dd388e2"}
Sep 11 14:06:08.538635 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:08.538556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676" event={"ID":"5df01421-67d6-4d46-ba73-dd1ab679a59b","Type":"ContainerStarted","Data":"7fac79f5d9576451ede9b257dfc19689df3b5570567bb4be1121a207db7a2be4"}
Sep 11 14:06:08.539002 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:08.538705    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:11.887072 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:11.887042    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9lc_5a39d72e-6a80-4205-93a5-8dd0567ba6b1/dns/0.log"
Sep 11 14:06:11.943170 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:11.943144    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cn9lc_5a39d72e-6a80-4205-93a5-8dd0567ba6b1/kube-rbac-proxy/0.log"
Sep 11 14:06:12.148119 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:12.148055    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tkfh5_ec4d4872-4d83-4d4a-a78f-6945f2447a76/dns-node-resolver/0.log"
Sep 11 14:06:13.149694 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:13.149669    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tq76r_93b4f034-59d0-442e-b948-370d656307d9/node-ca/0.log"
Sep 11 14:06:14.182807 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:14.182778    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6_baacab4d-9cb0-46af-bc3f-a65f7ce4d74b/istio-proxy/0.log"
Sep 11 14:06:14.551327 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:14.551296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676"
Sep 11 14:06:14.569807 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:14.569751    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwnld/perf-node-gather-daemonset-77676" podStartSLOduration=8.56973439 podStartE2EDuration="8.56973439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:09.097852544 +0000 UTC m=+2693.491539953" watchObservedRunningTime="2026-09-11 14:06:14.56973439 +0000 UTC m=+2698.963421826"
Sep 11 14:06:14.609119 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:14.609096    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-shkfw_e6491d9c-b73d-481a-8c82-f16de3694df4/discovery/0.log"
Sep 11 14:06:16.181533 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:16.181490    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nllgk_f4b75c12-9d04-448a-9285-b02523dd697e/serve-healthcheck-canary/0.log"
Sep 11 14:06:17.223775 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:17.223705    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fds4p_0da8220f-d89e-4f9a-a453-75b2a0947c5d/kube-rbac-proxy/0.log"
Sep 11 14:06:17.315159 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:17.315134    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fds4p_0da8220f-d89e-4f9a-a453-75b2a0947c5d/exporter/0.log"
Sep 11 14:06:17.384439 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:17.384416    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fds4p_0da8220f-d89e-4f9a-a453-75b2a0947c5d/extractor/0.log"
Sep 11 14:06:18.385559 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:18.385530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6_baacab4d-9cb0-46af-bc3f-a65f7ce4d74b/istio-proxy/0.log"
Sep 11 14:06:18.807544 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:18.807513    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-shkfw_e6491d9c-b73d-481a-8c82-f16de3694df4/discovery/0.log"
Sep 11 14:06:20.492658 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:20.492627    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-v4p74_6b43c580-f9a2-4dd4-9f5a-8c9e8336ccaa/server/0.log"
Sep 11 14:06:20.526919 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:20.526883    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cq6xb6_baacab4d-9cb0-46af-bc3f-a65f7ce4d74b/istio-proxy/0.log"
Sep 11 14:06:20.601151 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:20.601117    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kv2sr_4165f24d-343b-4bb9-8dd9-354b7e639e71/manager/0.log"
Sep 11 14:06:20.773902 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:20.773873    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-hc9dj_b2e85ee5-6a8e-47a6-b66c-fa554515c01b/manager/0.log"
Sep 11 14:06:20.806951 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:20.806928    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9576ccb85-hc9dj_b2e85ee5-6a8e-47a6-b66c-fa554515c01b/copy-manifests/0.log"
Sep 11 14:06:21.001514 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:21.001474    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-shkfw_e6491d9c-b73d-481a-8c82-f16de3694df4/discovery/0.log"
Sep 11 14:06:22.216563 ip-10-0-133-221 kubenswrapper[2576]: I0911 14:06:22.216540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84bdbdc8f4-wm8fb_74b0bbc3-c829-481c-9c6a-45e53a8dfb87/manager/0.log"